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Abstract 

This report focuses on two aspects of teacher supply and quality: (a) the under-representation of 
racial/ethnic minorities, especially African Americans in the teaching pool, and (b) teacher 
candidates’ performance on licensure assessments, including general skills tests in reading, 
writing, and mathematics (known as Praxis I ) and selected tests covering areas of content and 
pedagogy (known as Praxis II®). The report presents trends in the overall and minority teaching 
workforce, utilizes some of the available Praxis™ program data, combined with interviews of 
faculty and students involved in college and university teacher preparation programs, to address 
supply, achievement, and performance gaps between prospective minority and White teachers. 
African American candidates are the focal group in this report, in order to gauge the effect that 
Praxis has on the supply of prospective African American teachers and to generate ideas for 
closing gaps. Other minority Praxis test-taking populations in the selected states are not included 
because these are not large enough to produce sufficient data from which to make valid 
inferences. 

The research was collaboratively planned and guided by National Education Association 
and Educational Testing Service to use Praxis data for addressing the following questions: (a) 
What are the trends in minority representation among teacher candidates, (b) What are the 
differences between majority and minority candidates’ performance on both Praxis I and Praxis 
II, (c) Are the performance differences reflected in the state licensing pass rates of majority and 
minority candidates, and (d) What factors should be targeted to reduce racial/ethnic group 
performance differences on Praxis I and Praxis II. 

Key words: achievement gap, Praxis I, teacher licensure, Praxis II, passing scores 
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Introduction 

Like many specialized occupations in the United States, the teaching profession is 
confronting the enormous challenges of supply and quality. These challenges are 
multidimensional and include complexities that consume substantial attention of national, state, 
and local education policymakers. Policymakers address teacher supply and quality from a 
variety of competing perspectives and with varying interests. Included among the key supply 
challenges, for example, are the availability of people to (a) replace retiring teachers, (b) replace 
teachers who leave the profession at a young age in order to pursue alternative careers, (c) teach 
advanced courses in high-demand subjects such as the sciences and mathematics, (d) fill the 
highly demanding but least desirable jobs in disadvantaged schools and communities, and 
(e) contribute to reducing the under-representation of minorities among actively credentialed 
teachers. Included among the challenges of quality are the following: (a) ensuring a match 
between the field and discipline of a teacher’s training and the courses they are assigned to teach, 
(b) achieving diversity in the pool of people earning credentials by passing licensing, 
certification and career ladder examinations, (c) agreeing upon performance criteria for 
compensation, and (d) measuring the extent to which the students of each teacher achieve 
learning and performance standards established by the district, state, and the nation. 

This report focuses on two aspects of teacher supply and quality: (a) the under- 
representation of racial/ethnic minorities, especially African American teachers, and (b) teacher 
candidates’ reading, writing and mathematics skills, as represented by candidates’ performance 
on Praxis™. The report satisfies an agreement between the National Education Association 
(NEA) and Educational Testing Service (ETS) to make use of some of the available data 
generated by the Praxis program, combined with information gathered by interviewing faculty 
and students who are involved in college and university teacher preparation programs, to address 
achievement and performance gaps between prospective minority and White teachers. African 
American candidates are the focal group in this report, in part because other minority Praxis test- 
taking populations in the states that were selected were too small to include. 

NEA and ETS are involved in different but complementary aspects of teacher supply and 
quality. NEA seeks to ensure that the nation’s schools have an adequate supply of qualified 
teachers, and ETS seeks to ensure that the academic achievements of teachers who enter the 
profession correspond with the knowledge, skills, and standards established by the states. 
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Performance on Praxis is used by the majority of states as one of several criteria for licensing 
new teachers and by many colleges and universities to admit students into teacher education 
degree programs. In 2006, NEA and ETS embarked on this project to examine candidates’ 
performance on assessments of general knowledge and skills known as the Praxis I® Pre- 
Professional Skills Tests (PPST®) and selected specialty tests (known as Praxis II® Subject 
Assessments) in part to see if these factors contributed to the undersupply of minority teachers. 

The following questions guided the research presented in this report: 

1. What are the trends in minority representation among teachers? 

2. How much of a difference exists between majority and minority candidates’ 
performance on both Praxis I and Praxis II? 

3. Are the performance differences reflected in the state licensing pass rates of majority 
and minority candidates? 

4. What factors should be targeted in attempts to eliminate racial/ethnic group 
performance differences on Praxis I and Praxis II? 

Before proceeding to the analyses of group performance differences on Praxis and the 
effects of Praxis on the supply of minority teachers, we present an analysis of trends in the 
overall and minority teaching workforce. In the end, the trend analyses taken together with the 
analyses of performance gaps and passing rates and the correlates of Praxis performance should 
help gauge the effect that Praxis has on the supply of prospective African American teachers and 
generate ideas for closing gaps. 

Overall and Minority Trends in Teacher Supply in the United States 

The overall supply of the nation’s teacher workforce has been growing in size and 
diversity in recent years. Table 1 presents a summary of the positive trends for two points in time 
from 1994 to 2004 (Planty et al., 2007). 

During that decade, while elementary and secondary school enrollments were increasing 
in the United States by 10%, the number of full-time teachers grew by nearly 20%. The 
demographic composition of the nation’s teacher supply also grew younger and more racially 
and ethnically diverse. The number of teachers from under-represented minority groups 
increased as well over that period. During a time when the growth of students from minority 
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groups was steady, the teaching profession achieved more dramatic progress in attracting a larger 
number of younger and racial/ethnic minority teachers. Despite the gains, however, African 
American, Hispanic, and Native American populations remained severely under-represented in 
the teaching profession relative to the population of students. In 2003-2004, under-represented 
minority teachers made up only 17% of the public school teaching force, while minority students 
made up 42% of the public school enrollments (National Center for Education Statistics, 2006, 
2007). 

Table 1 



Trends in Teacher Supply and Demographics From 1994 to 2004 



Category 


1994 


2004 


Teacher supply 


Full-time teachers 


2.6 million 


3.1 million 


Teacher age and experience 


Proportion under age 30 


12% 


17% 


Proportion with 3 or fewer years experience 


13% 


16% 


Proportion with 4-10 years experience 


21% 


27% 


Teacher race/ethnicity 


African American 


7% 


8% 


Hispanic 


4% 


6% 


Native American 


0.7% 


0.5% 



Among the possible explanations for the continuing under-representation of some 
minority groups are the relatively low production of new baccalaureate degree recipients with an 
interest in teaching and the relatively low success rate of prospective teachers on the tests 
required for licensure. 

Overall and Minority Bachelor’s Degree Production in Education 

After peaking in 1991 and leveling off in 1992 and 1993, both the number of education 
majors and the proportion that they comprised of total bachelor’s degree recipients declined a bit 
in 1994 and then remained constant through 2006. During the same period, however, the overall 
proportion of bachelor’s degrees awarded to under-represented minorities increased as did their 
representation of total bachelor’s degrees awarded. In 1994, of all bachelor’s degrees awarded in 
the United States, African American students represented 7% of degree recipients, Hispanic 
students 4%, and Native American students 0.5%. At the same time, of all bachelor’s degrees 
awarded in education, African American students comprised 5%, Hispanic students 3%, and 
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Native American students 0.6% (Snyder, Dillow, & Geddes, 1996, Table 261). In 2006, the 
number of African American bachelor’s degree recipients grew to around 10% of the nearly 1.5 
mi llion total bachelor’s degrees awarded (all fields), Hispanic recipients to around 7%, and 
Native American recipients to 0.7%. Yet, African American bachelor’s degrees recipients in 
education were only 6% of all bachelor’s degrees in education, and Hispanic recipients were 5%. 
At 0.8%, Native American representation among degree recipients in education was roughly 
equal to their overall representation (Snyder, Dillow, & Hoffman, 2007, Table 275). 

While the colleges and universities that annually award bachelor’s degrees in the field of 
education continue to be the largest sources of new teachers, they are not the sole sources. Emily 
Feistritzer, president and founder of the National Center for Education Information (NCEI), 
estimated that around 35,000 new teachers each year now enter through alternative routes, and 
minorities comprise a much larger share of the alternative route population than they represent 
among bachelor’s degree recipients in the field of education (NCEI, 2008). Alternative routes to 
teaching represent a large and growing enterprise, producing one third as many new teachers 
each year (approximately 108,000) as are produced by college and university bachelor’s degree 
programs in the field of education (Planty et al., 2008). Consequently, the overall numbers of 
alternate route teachers, coupled with the apparent over-representation of minority alternate route 
teachers, suggests that alternative routes are making a substantial contribution to increasing the 
minority teacher supply. 

Regardless of whether entry into the teaching system occurs through conventional or 
alternative routes, licensure examinations like Praxis are prominent in the screening and quality 
control processes. The extent to which the supply is affected by assessments used for admissions 
into teacher training programs, for degree qualification, and for earning licenses to practice is 
rather easy to judge at the local level and, therefore, may be a factor in setting admissions 
requirements for teacher preparation programs and cut scores for licensing. But because of the 
variety of requirements, standards, assessments/tests, and sources of teachers throughout the 
nation, it is more difficult to ascertain the effect of assessments nationally. Fully understanding 
the relationship of assessments to the supply, quality, and demographic composition of the 
teaching workforce requires starting with an examination of all assessments used for these 
purposes. We had access to one, the Praxis series of assessments, and conducted analyses and 
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used these findings as a means of introducing the process for such inquiries and to examine the 
issue in states where Praxis is being used. 

Research Questions 

The analyses in this report focused on group differences, primarily between African 
American and White prospective teachers, in performance on Praxis. The following questions 
were addressed: 

• What are the performance differences between African American and White test- 
takers on basic skills and selected subject areas of Praxis? 

• How are the background characteristics of Praxis examinees associated with 
performance and the performance differences between African American and White 
examinees? 

• How do college and university faculty who are involved in preparing new teachers 
perceive the challenges to their students on Praxis examinations? 

• What efforts are college and university faculty making to meet the examination 
challenges of minority test-takers? 

The remainder of this report includes analyses of Praxis I and Praxis II tests and the 
related background characteristics of examinees, findings from interviews of students and faculty 
of teacher education programs at various minority- serving colleges and universities, and 
concluding commentary and recommendations. 

Data Source and Sample 

Rather than reviewing all 133 existing Praxis tests, ETS and the NEA selected 16 paper- 
and-pencil tests with relatively large volumes of test-takers. 1 These tests included the three PPST 
(or Praxis I) and 13 selected Praxis II tests in a variety of content areas. The sample for analyses 
included people who tested between November 2005 and November 2009, which spanned a total 
of 29 test administrations (20 for Praxis I). 

Findings From Praxis I Analyses 

The pool of Praxis I candidates was based on data from the 28 states where Praxis I was 
administered between November 2005 and November 2009 and a valid passing score was 
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documented. Nine of these states had unique requirements for licensure on all three Praxis I tests. 
These requirements involved using a composite score methodology consisting of an aggregate of 
the three Praxis I test scores. The distinction within these requirements was that some of these 
states did not require minimum scores on each of the three Praxis I tests as long as a specified 
composite score was reached. By contrast, other such states required both a minimum score on 
each Praxis I test and a minimum composite score. Nonetheless, candidates from these states 
were included in the analyses. 

One key aspect of this study was that it included a sample of Praxis I first-time test- 
takers. In addition to test scores, the personal information that candidates provided on the 
Background Information Questionnaire ( B IQ) - when they registered to take the tests comprised 
the core data for the analyses. Table 2 presents the overall population of initial Praxis I test- 
takers in each race/ethnicity group spanning November 2005 to November 2009. The total 
number of test-takers in the 28 states between November 2005 and November 2009 across the 
five prominent race/ethnicity groups of test-takers (White, African American, Hispanic, Asian 
American, and Native American) was about 133,000 for each of the three skills tests of reading, 
writing, and mathematics. The sample was pared down to select the first-time test-takers for each 
test who requested that their scores be submitted to a state department of education. First-time 
test-takers comprised roughly 60% of the total test-takers in the 28 states that administered 
Praxis I in the timeframe covered in this report. 



Table 2 

Sample Sizes and Percentages of All Praxis I First-Time Test-Takers (FTTTs) Between 
November 2005 and November 2009 by Race/Ethnicity 





Praxis I Reading 




Praxis I Writing 




Praxis I Mathematics 




Race/ethnicity 


Total 


% 


FTTTs 


% 


Total 


% 


FTTTs 


% 


Total 


% 


FTTTs 


% 


White 


104,378 


80 


65,782 


83 


103,997 


79 


65,792 


84 


103,108 


78 


64,637 


84 


African 


American 


16,940 


13 


8,408 


11 


14,998 


11 


8,213 


10 


16,326 


12 


8,117 


11 


A sian 


American 


5,594 


4 


2,251 


3 


5,026 


4 


2,244 


3 


4,901 


4 


2,198 


3 


Hispanic 


3,358 


3 


1,901 


2 


3,311 


3 


1,909 


2 


3,472 


3 


1,887 


2 


Native 


American 

Total 


863 

131,133 


1 


450 

78,792 


1 


889 

128,221 


1 


457 

78,615 


1 


862 

128,669 


1 


435 

77,274 


1 



Note. Percentages may not add up to exactly 100% due to rounding. 
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African American and White candidates made up the vast majority (about 95%) of the 
first-time test-takers in the 28 states, 11% and 84%, respectively. Due to the smaller numbers of 
Hispanic, Asian American, and Native American test-takers compared to African American and 
White test-takers, the focus of the analyses in this report was on the latter two groups of test- 
takers. In part, the relatively small representation of Hispanic and Asian American test-taking 
populations in this study reflected their low representation in states that require Praxis I. At about 
11%, African American candidates in these 28 states represented a larger share of the first-time 
test-takers than they represented in the nation’s teaching workforce in 2004 (8%) and even 
among baccalaureate degree recipients in the field of education in 2006 (10%). 

Rather than limiting the sample to people who took all three Praxis I exams, we included 
everyone who took any of the three Praxis I tests for the first time. Approximately 8% of the 
examinees during the time frame of these analyses did not take all three Praxis I exams. To check 
if limiting our sample to people who took all three Praxis I exams made a difference, we 
conducted analyses comparing the mean scale scores of the 92% of our sample who took all 
three exams with the 8% of our sample who took only one or two of the three exams. No 
substantive differences in mean scale scores were detected. Therefore, the authors felt that the 
findings from any analyses using the individual test populations would extend to this common 
cohort representing 92% of our sample. Additionally, the research questions addressed individual 
Praxis I exams, so the common cohort of test-takers was not used for the analyses in this report. 
The population of first-time test-takers on the three Praxis I tests in the specified timeframe, and 
included for this report, ranged from 72,754 to 74,190 candidates, about 84% of whom were 
White and about 1 1% of whom were African American. 

Descriptive analyses of African American- White candidate Praxis I score 
differences. The 28 states included in this study employ much the same approach to setting the 
passing score point (commonly referred to as the cut score). The standard setting methodology 
that they employ is presented in Appendix A. Every state independently chooses where to set the 
cut score on each test. For the present study, this decision was important because we examined 
both the differences in the scores between African American and White candidates on Praxis I 
and the consequences of these differences on the rates at which these candidates qualify for a 
state’s license/credential to teach. For purposes of this study, if a candidate submitted scores to 
more than one state department of education for one administration and achieved the minimum 
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passing score in at least one state, we considered him or her to have passed that Praxis test. 

While Praxis I test scores are all reported on the same score scale of 150 to 190, the ranges of cut 
scores across the 28 states differ slightly: Praxis I Reading (170-178), Praxis I Writing (171— 
176), and Praxis I Mathematics (169-178). In each of these 28 states, students are required to 
pass each of the Praxis I exams to fulfill the testing requirements on the path to licensure. Total 
scale scores were the dependent variables in all analyses in this report. 

Table 3 presents the average scale scores for White and African American candidates on 
each of the three Praxis I exams with the differences expressed in standard deviation units, 4 also 
known as a standardized difference or effect size, sometimes represented by d (Cohen, 1988). 5 

Table 3 



Praxis I Mean Scale Scores by Race/Ethnicity With Gaps Presented in Standardized Terms 



Praxis I test 




White (W) 
candidates 




African American (AA) 
candidates 


Std. 

gap 


N 


Mean 


SD 


N 


Mean 


SD 


Reading 


65,782 


178.03 


5.43 


8,408 


171.61 


7.08 


-1.14 


Writing 


65,792 


175.96 


4.17 


8,213 


171.97 


4.23 


-0.95 


Mathematics 


64,637 


178.59 


6.89 


8,117 


170.56 


7.31 


-1.16 



Note. Std. gap = the standardized gap in mean scores between African American and White 
candidates (AA - W). 



Because of the constricted scales used for the Praxis tests, the scale score differences 
may give the impression that differences are very small (4 to 8 points), and smaller than we are 
accustomed to seeing on large-volume standardized tests used by colleges and universities for 
assessing general skills for admission like the SAT® and GRE®. Camara and Schmidt (1999) 
examined African American-White test-taker score gaps in standardized terms on several 
different tests, as displayed in Table 4. The range was from 0.82 (ACT Reading) to 1.14 
(LSAT). 

When applied to the analyses of score gaps on the Praxis I tests presented in Table 3, 
rather large differences between African American and White test-takers are revealed. The data 
presented in Table 3 suggest that the Praxis I Reading score gap (1.14 SDs ) between White and 
African American candidates is larger than that of the SAT and more like that for the GRE 
Analytical section. The gap in Praxis I Writing (0.95 SDs) is more comparable to the SAT, and 
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the gap in Praxis I Mathematics (1.16 SD s) is larger than that for the GRE Quantitative section. 
In terms of effect sizes, all standardized score differences on Praxis I tests are considered to be 
large (i.e., greater than 0.80). 



Table 4 

African American-White Test-Taker Performance Differences in Standardized Terms on 



Selected Large-Volume Standardized Tests 



Exam 


Standardized difference 


SAT Verbal 


-0.83 


SAT Math 


-0.92 


ACT English 


-0.89 


ACT Mathematics 


-0.88 


ACT Reading 


-0.82 


GRE Verbal 


-0.96 


GRE Quantitative 


-0.98 


GRE Analytical 


-1.11 


GMAT Total 


-1.03 


LSAT 


-1.14 


MCAT Verbal Reasoning 


-0.96 



For licensure tests, which are pass/fail, group performance differences may not be much 
of an issue if the pass rates for the groups are not very different. In the case of Praxis I licensure 
assessments, however, the pass rates are indeed a reflection of the African American-White test- 
taker score gap. Table 5 shows that African American first-time test-takers had a significantly 
lower pass rate than White first-time test-takers on each Praxis I exam. The Praxis I Mathematics 
exam, which had the largest score gap, also had the largest gap in pass rate. 

Table 5 



Differences in Passing Rates on Praxis I Tests by Race/Ethnicity Group 





Reading 


Writing 


Mathematics 


Percentage of first-time African 
American test-takers who passed 


40.7 


44.2 


36.8 


Percentage of first-time White 


81.5 


79.5 


78.2 


test-takers who passed 



Correlates of performance on Praxis I tests derived from the Background 
Information Questionnaire (BIQ). Beyond describing the differences in the Praxis I scores and 
passing rates between African American and White test-takers, the analyses were also intended 
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to identify characteristics and behaviors of test-takers that might be targeted by people who are 
making efforts to close the gaps. Several independent variables were selected based on four 
criteria: (a) their historic use in predicting performance on standardized tests (Messick, 1995), 

(b) their typical value and importance in helping to explain Praxis I test performance (Blue, 
O’Grady, Toro, & Newell, 2002; Sadovnik et al., 2008), (c) their typical value in explaining 
score differences among race/ethnicity groups and between gender groups (Nettles, Millett, & 
Einarson, 2001; Nettles, Millett, & Ready, 2003; Nettles, Pema, & Millett, 1998), and (d) then- 
availability in the Praxis data system from test-taker responses on the BIQ. 

The background variables available in the BIQ that were included in the analyses were the 
following: (a) undergraduate grade point average (UGPA), (b) enrollment status in a teacher 
education program, (c) candidate educational attainment, (d) socio-economic status (SES), as 
represented by the highest educational attainment of either the candidate’s mother or father, 

(e) undergraduate major, classified into general interest areas of science, business, social science, 
education, and humanities, and (f) college/university selectivity, as measured by Barron ’s Profile 
of American Colleges (2009). Detailed descriptions of these variables are presented in Appendix B. 

We calculated the proportions of candidates at each possible response level or range for 
each BIQ variable. Some categories for some BIQ variables were consolidated or omitted due to 
sample size considerations. We also calculated the standardized scale score gaps for each of the 
Praxis I tests for the BIQ variables at each possible response level or range. We also conducted 
regression analyses with selected BIQ variables to measure their influences as predictors of 
Praxis I performance. We specified race/ethnicity to interact with the other variables to assess 
how the relationship of these variables on Praxis I exam scores vary by race/ethnicity. Formulas 
for the statistical analyses are presented in Appendix C and details of the regression analyses are 
presented in Appendix D. 

Undergraduate grade point average (UGPA). Table 6 presents comparisons of mean 
Praxis I scores arrayed by UGPA ranges of African American and White test-takers along with 
the standardized gaps by test. As UGPAs increase, so, too, do mean Praxis I scores. It is 
interesting that the score gaps follow the same pattern. As the UGPAs increase, the score gaps 
grow wider, such that the largest Praxis I score gaps are found among test-takers in the highest 
UGPA ranges. 



10 




Table 6 



Praxis I Performance by Race /Ethnicity Group and Undergraduate Grade Point Average (UGPA)for First-Time Praxis I Test- 
Takers 



UGPA 


Frequency 






Reading 








Writing 








Mathematics 




W 


AA 


W 


AA 




W 


AA 




W 


AA 




% 


% 


M 


SD 


M 


SD 


d 


M 


SD 


M 


SD 


d 


M 


SD 


M 


SD 


r/ 


1.5-1.99 


< 1 


1 


173 


7 


169 


1 


-0.60 


173 


3 


171 


4 


-0.55 


175 


1 


169 


7 


-0.94 


2.0-2.49 


4 


13 


175 


6 


170 


7 


-0.86 


174 


4 


171 


4 


-0.73 


176 


7 


169 


7 


-0.92 


2.5-2.99 


22 


36 


176 


6 


171 


7 


-0.87 


174 


4 


171 


4 


-0.78 


177 


7 


170 


7 


-0.91 


3.0-3.49 


39 


34 


178 


5 


172 


7 


-1.00 


176 


4 


172 


4 


-0.81 


178 


7 


171 


7 


-1.09 


3. 5-4.0 


35 


16 


180 


5 


173 


7 


-1.41 


178 


4 


173 


4 


-1.04 


181 


6 


172 


8 


-1.36 



Note. AA = African American, W = White, UGPA = undergraduate grade point average. 




Table 6 also reveals that the score gaps are larger in mathematics than in reading and 
writing overall and within comparable UGPA ranges. One indication of the large overall gap in 
mathematics is revealed by comparing scores across groups throughout the distribution of UGPA 
ranges. Among mathematics test-takers, the mean scale score of African American test-takers in 
each UGPA range, including the highest range (3.5 - 4.0), was lower than the mean Praxis I 
score in the lowest UGPA range (1.5 - 1.99) for White test-takers. The mean score for African 
American mathematics test-takers in the highest UGPA range was 172 compared to 175 for 
White test-takers in the lowest UGPA range. In reading, where the gaps were second largest 
among the three Praxis I tests, the mean score of African American test-takers in the highest 
UGPA range was below the White test-taker mean score in each of the other UGPA ranges with 
the exception of the lowest range. In both reading and writing, African American candidates in 
the highest UGPA range achieved an average Praxis I score of 173, equal to that among White 
candidates in the lowest UGPA range. 

Part of the reason for the increasing score gap at successively higher levels of UGPA was 
that the rate of increase in the scores of African American candidates as UGPAs increased was 
not as large as the rate of increase in scores of their White counterparts. For example, in reading, 
the difference in average scores between the lowest and highest UGPA categories among African 
American test-takers was about 4 points, while for White test-takers it was 7 points. The 
differences between the highest and lowest UGPA categories were 2 and 5 points for African 
American and White candidates, respectively in writing, and 3 and 6 points, respectively in 
mathematics. 

Enrollment status in teacher education program. Like UGPA, the racial/ethnic 
differences by teacher education degree program enrollment status are also interesting. The 
teacher education program enrollment frequencies across the different status indicators 
(currently, formerly, never) varied between African American and White Praxis I test-takers. 
About 54% of White test-takers compared to 37% of African American test-takers were 
currently enrolled in such programs at the time of taking the test, roughly 8% of White 
candidates compared to 14% of African American candidates were formerly enrolled, and 37% 
of White test-takers compared to 49% of African American test-takers had never been enrolled. 

The large proportion of candidates who had never been enrolled in a teacher education 
program can be explained in part by four factors: (a) the tests are taken as part of entry 
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requirements for a teacher education program, which means that many candidates are taking the 
Praxis I exams as part of the admissions process; (b) the tests are taken by candidates who have 
established careers in fields other than teaching and are taking the Praxis I exams as part of their 
career change process to enter the teaching profession as undergraduates; (c) some states have 
waivers for Praxis I tests, whereby an SAT or ACT performance above a certain level is 
sufficient to exempt students from taking Praxis I as a requirement for entry into teacher 
education programs; and (d) the increasing presence of alternate route certification programs 
allows many students to enter the teaching profession with college major fields other than 
education. A question was added to the BIQ in July 2008 to inquire about the nature of a 
candidate’s teacher preparation program. The large majority of candidates went through an 
undergraduate teacher education degree program (74%). A preliminary estimate suggested that 
approximately 8% of candidates in the Praxis I samples indicated preparation through an 
alternate route program. This proportion was only slightly less than that for those going through 
a master’s degree program (9%). More data are needed to substantiate this finding. 

The data presented in Table 7 tell a mixed story, and one that is not intuitive. There 
appeared to be no clear effect of enrollment status in a teacher education program on the gap 
between race/ethnicity groups. For all Praxis I tests, the gap between African American and 
White test-takers appeared smallest for those who had never enrolled in a teacher education 
program. Only on the mathematics exam did the gap decrease slightly for those formerly 
enrolled in a program compared to current enrollees. 

Candidate educational attainment level. Table 8 presents the frequency distributions of 
test-takers by educational attainment level and the corresponding Praxis I mean scores and score 
gaps between African American and White test-takers. White candidates tended to be 
underclassmen when taking Praxis I while African American candidates were more often 
upperclassmen when taking Praxis I. Slightly more than half of the White test-takers (52%) 
compared to slightly more than a quarter of the African American test-takers (24%) were either 
freshmen or sophomores when they took the Praxis I tests. Additionally, our calculation of the 
age of test-takers using the test date and test-taker date of birth, both provided in the BIQ, 
revealed that African American prospective teachers were first taking Praxis I later in life (just 
over 30 years of age) than their counterpart White test-takers (about 25 years of age). 
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Table 7 



Praxis I Performance by Race/Ethnicity Group and Enrollment Status in Teacher Education Programs for First-Time Praxis I 
Test-Takers 



Status 


Frequency 






Reading 








Writing 








Mathematics 




W 


AA 


W 


AA 




W 




AA 




W 




AA 




% 


% 


M 


SD 


M 


SD 


d 


M 


SD 


M 


SD 




M 


SD 


M 


SD 


d 


Currently 


54 


37 


178 


5 


171 


7 


-1.20 


176 


4 


172 


4 


-0.98 


179 


1 


170 


1 


-1.23 


Formerly 


8 


14 


179 


6 


171 


7 


-1.44 


111 


5 


172 


5 


-1.05 


178 


1 


170 


1 


-1.19 


Never 


37 


49 


178 


6 


172 


7 


-1.00 


176 


4 


172 


4 


-0.90 


178 


1 


171 


1 


-1.06 



Note. AA = African American, W = White. 



Table 8 



Praxis I Performance by Race/Ethnicity Group and Test-Taker Educational Attainment Level for First-Time Praxis I Test-Takers 





Frequency 






Reading 










Writing 








Mathematics 






Education 


W 


AA 


W 




AA 






W 




AA 






W 




AA 






level 


% 


% 


M 


SD 


M 


SD 


d 


M 


SD 


M 


SD 


d 


M 


SD 


M 


SD 


d 


Freshman 


24 


8 


178 


5 


173 


1 


-0.88 


176 


4 


173 


4 


-0.83 


180 


6 


174 


1 


-0.98 


Sophomore 


28 


16 


178 


5 


172 


1 


-0.96 


176 


4 


172 


4 


-0.84 


179 


7 


172 


1 


-1.01 


Junior 


15 


18 


111 


6 


171 


1 


-1.01 


175 


4 


172 


4 


-0.84 


111 


7 


170 


1 


-0.98 


Senior 


9 


14 


111 


6 


170 


1 


-1.15 


175 


4 


171 


4 


-0.93 


111 


7 


170 


1 


-1.03 


Bachelor’s 

degree 


10 


19 


180 


5 


172 


1 


-1.38 


111 


4 


172 


4 


-1.09 


179 


7 


170 


1 


-1.19 


Bachelor’s 
degree + 


9 


17 


180 


5 


172 


1 


-1.60 


111 


4 


172 


4 


-1.14 


179 


7 


170 


1 


-1.25 


Master’s 

degree 


2 


5 


181 


5 


172 


1 


-1.67 


178 


4 


172 


5 


-1.25 


180 


7 


170 


1 


-1.35 


Master’s 
degree + 


2 


3 


182 


4 


173 


1 


-1.69 


179 


4 


174 


5 


-1.11 


180 


7 


171 


1 


-1.26 



Note. AA = African American, W = White. 




The data presented in Table 8 resemble those shown in Table 6 with UGPA where the gap 
appears to widen as educational attainment level increases for each of the three Praxis I tests. At 
levels beyond the baccalaureate degree, the gaps are wider on the reading and mathematics tests. 

Even though educational attainment level is associated with score differences on Praxis I, 
it could be that the relationship of educational attainment level differences to score differences is 
masking the age by educational attainment level differences between the two groups. While the 
average age of first-time test-takers among White candidates was about 5 years younger (about 
age 25) than African American candidates (about age 30), the differences in average ages of 
first-time test-takers between the two racial/ethnic groups within the four highest education 
attainment levels were smaller. For examinees with bachelor’s degrees, African American 
candidates first took Praxis I about 4 years later (about age 34) than White candidates (about age 
30), while for those with at least a bachelor’s degree but who had not yet received a master’s 
degree, the difference was around 3 years (about age 37 for African American candidates; about 
age 34 for White candidates). Beyond that level of degree attainment, the differences in test- 
takers’ ages were trivial, so another factor like educational attainment level may be confounded a 
bit by age in explaining larger gaps on Praxis I tests. However, the data clearly showed that an 
association exists between age and educational attainment level. 

At the higher education attainment levels (bachelor’s degree and higher) for both 
racial/ethnic groups, a chi-square test detected a relationship (p < .001) between educational 
attainment level and enrollment status in a teacher education program. The data suggested that 
African American candidates with at least a master’s degree taking Praxis I underperformed 
more compared to their White peers if they were then enrolled in a teacher education program, as 
effect sizes were higher (-1.74 for reading; -1.27 for writing; -1.36 for mathematics) than those 
reported in Table 8 for master’s or beyond master’s degree recipients across teacher education 
program enrollment status levels. These statistics were also true for those who were formerly 
enrolled in such programs, but not so for those never enrolled. The preparation of teacher 
education program candidates is examined from a qualitative perspective later in this report. 

Socio-economic status (SES). The variable for SES in this study was the highest 
educational attainment of either parent. Table 9 presents African American and White test-taker 
mean scores on Praxis I Reading, Writing, and Mathematics along with the gaps in the scores 
arrayed by level of parental educational attainment. As expected, White test-takers were better 
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represented among categories of higher parental educational attainment beyond the baccalaureate 
degree than their African American counterparts. For example, around 21% of White test-takers’ 
parents attained a graduate or professional degree compared to 15% of African American test- 
takers’ parents. Roughly the same representations and differences existed between White and 
African American test-takers’ parents (25% and 17%, respectively) who completed 
baccalaureate degrees. Conversely, a larger share of the parents of African American test-takers 
had completed a high school diploma or less than their White counterparts: 36% compared to 
24%. Consistent increases in gaps occurred in writing and mathematics as SES level increased, 
except at the level of high school diploma. 

Undergraduate major field. Table 10 presents in succession the Praxis I Reading, 

Writing, and Mathematics mean scores and gaps between African American and White test-takers 
in the sample arrayed by selected broad major field of study. The proportions of White and African 
American candidates who were science majors were comparable at 7%, and the gaps were among 
the largest across the three Praxis I tests. White and African American humanities majors were 8% 
and 7% of their respective group’s test-takers and the gap was smallest in writing. 

Nearly three quarters of White test-takers (72%) and two thirds of African American test- 
takers (67%) majored in education. Education majors achieved the lowest mean scores on each 
of the Praxis I tests, and the gaps in reading and mathematics, while still large, were only slightly 
narrower than for other majors. The gap in writing among education majors was similar to that 
for social science and humanities majors, and slightly lower than that for other majors. 

The second largest representation of both African American and White test-takers 
majored in social sciences at 12% and 11%, respectively. Similar to the sciences, a small 
proportion of examinees majored in business, but a somewhat larger share of African American 
candidates (6%) than Whites (2%) were business majors. The gaps in these two major fields are 
not distinctive, and taken altogether, the score differences among the major fields raises 
questions about the extent to which curricula influence test performance. We present the 
curricular influence later in the report. 
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Table 9 



Praxis I Performance by Race/Ethnicity Group and Socio-Economic Status (SES)for First-Time Praxis I Test-Takers 





Frequency 






Reading 








Writing 








Mathematics 






W 


AA 


W 




AA 




W 




AA 




W 




AA 




SES 


% 


% 


M 


SD 


M 


SD 


d 


M 


SD 


M 


SD 


d 


M 


SD 


M 


SD 


d 


< Some HS 


2 


10 


111 


6 


169 


1 


-1.15 


174 


4 


171 


4 


-0.78 


175 


1 


168 


1 


-0.97 


HS diploma 


22 


26 


111 


6 


170 


1 


-1.09 


175 


4 


171 


4 


-0.95 


111 


1 


170 


1 


-1.14 


Some post 
sec. ed. 


14 


17 


178 


5 


173 


1 


-1.05 


176 


4 


172 


4 


-0.85 


179 


1 


172 


8 


-1.03 


Assnriatp’s 


degree 


10 


11 


111 


6 


171 


1 


-1.07 


176 


4 


172 


4 


-0.85 


178 


1 


170 


7 


-1.12 


Rnrhf'lnr’s 


degree 


25 


17 


178 


5 


173 


1 


-1.03 


176 


4 


173 


4 


-0.86 


179 


1 


171 


7 


-1.14 


Some 


graduate or 

professional 

school 


5 


5 


179 


5 


173 


1 


-1.10 


111 


4 


172 


4 


-1.00 


179 


1 


171 


7 


-1.20 


Graduate/ 


professional 

degree 


21 


15 


179 


5 


173 


1 


-1.17 


111 


4 


173 


4 


-1.00 


180 


1 


171 


7 


-1.26 



Note. AA = African American, HS = high school, W = White. 




Table 10 

Praxis I Performance by Race/Ethnicity Group and Undergraduate Broad Major Field Classification for First-Time 
Praxis I Test-Takers 





Frequency 






Reading 










Writing 








Mathematics 






W 


AA 


W 




AA 






W 




AA 






W 




AA 






Major 


% 


% 


M 


SD 


M 


SD 


d 


M 


SD 


M 


SD 


d 


M 


SD 


M 


SD 


d 


Science 


7 


7 


180 


5 


173 


1 


-1.45 


111 


4 


172 


4 


-1.08 


183 


6 


175 


8 


-1.23 


Business 


2 


6 


180 


5 


173 


1 


-1.31 


176 


4 


172 


4 


-1.01 


180 


1 


112 


7 


-1.22 


Social 

sciences 


11 


12 


180 


5 


174 


1 


-1.28 


111 


4 


173 


4 


-0.96 


179 


1 


170 


7 


-1.23 


Education 


72 


67 


177 


6 


171 


1 


-1.14 


175 


4 


171 


4 


-0.98 


178 


1 


170 


7 


-1.19 


Humanities 


8 


7 


181 


4 


174 


1 


-1.34 


178 


4 


174 


5 


-0.96 


179 


1 


171 


7 


-1.21 



Note. Those majoring in technology-related disciplines, those who were undecided, or those whose majors did not fit the five major 
groupings displayed in this table were removed from this analysis. AA = African American, W = White. 




Selectivity of colleges and universities attended. The selectivity of the colleges and 
universities that candidates attend can be a useful prism through which to view candidates’ 
scores and make racial/ethnic group comparisons on Praxis I scores for two reasons. First, given 
that Praxis I is a series of basic skills tests, it is more likely that test-takers attending more 
selective colleges and universities would achieve higher scores on Praxis I than their 
contemporaries attending less selective institutions. Second, the differing proportions of students 
between racial/ethnic groups attending more or less selective colleges and universities may be 
contributing to the overall African American-White test-taker score gaps on Praxis I. 

Barron ’s Profile of American Colleges (2009) compiled a list of 4-year colleges and 
universities in the United States according to their degree of selectivity, ranging from 
noncompetitive to most competitive. Due to low sample sizes near the upper extremes of 
Barron’s nine available categories, the four highest categories were consolidated for our 
analyses. Table 11 shows that the majority of African American and White test-takers attended 
midrange selective (competitive) colleges and universities, but a slightly larger proportion of 
White candidates in the sample were in that category of institutions. It is interesting, although not 
surprising, that a much smaller share of African American test-takers (14%) attended colleges 
and universities in the two most selective categories than White test-takers (28%) and that a 
larger share of African American test-takers attended schools in the two least selective 
categories: 30% compared to 15%. 

Table 11 also presents the mean scores and gaps by degree of selectivity of the colleges 
and universities candidates attended. The pattern for both African American and White test- 
takers on each of the three Praxis I tests was that the more selective the colleges and universities, 
the higher the mean scores were. Consistent with analyses earlier and throughout the report, the 
largest overall gaps throughout the spectrum of selectivity of attended institutions generally 
appeared to be in mathematics, and the largest gaps were among candidates attending colleges 
and universities at higher levels of selectivity (competitive to very competitive, inclusive). 
Otherwise, there appeared to be no vast differences or patterns in the sizes of the gaps across the 
distributions of test-takers based upon the selectivity of the college or university they attended 
compared to the overall gaps discussed in this report. 



19 




Table 11 



Praxis I Performance by Race/Ethnicity Group and Selectivity of the College or University for First-Time Praxis I Test-Takers 



Selectivity 


Frequency 






Reading 








Writing 






Mathematics 




W 


AA 


W 


AA 


d 


W 


AA 


d 


W 


AA 


d 


% 


% 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


Noncompetitive 


4 


5 


111 


6 


170 


1 


-1.00 


175 


4 


171 


4 


-0.81 


111 


1 


169 


6 


-1.20 


Less competitive 


11 


25 


111 


6 


171 


1 


-1.08 


175 


4 


171 


4 


-0.91 


178 


1 


170 


1 


-1.08 


Competitive 


51 


49 


178 


6 


172 


1 


-1.00 


176 


4 


172 


4 


-0.86 


178 


1 


171 


1 


-1.05 


Competitive + 


6 


6 


178 


5 


171 


1 


-1.42 


176 


4 


172 


5 


-1.09 


179 


6 


170 


1 


-1.41 


Very competitive 


19 


9 


179 


5 


174 


6 


-1.11 


177 


4 


173 


4 


-0.89 


180 


7 


172 


8 


-1.24 


Very competitive + 


9 


5 


181 


4 


176 


6 


-1.03 


179 


4 


175 


4 


-0.96 


182 


6 


174 


8 


-1.17 



Note. AA = African American, W = White. 
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Regression model summary. The descriptive and comparative analyses suggested that 
the differences in the Praxis I scores of White and African American test-takers may be at least 
in part related to their differences in UGPA, teacher education program enrollment status, their 
own educational attainment, their parental educational attainment, undergraduate major, and 
selectivity of attending institution. The next issue addressed was the extent to which these factors 
were associated with candidates’ performances and the relative weights that these had in 
predicting Praxis I scores. 

We conducted regression analyses to ascertain how candidates’ characteristics 
contributed to their performance on each of the three components of Praxis I and whether these 
relationships were different for African American and White test-takers. Given that each BIQ 
variable consists of multiple categories, these were consolidated to aid in conducting and 
interpreting the regression analyses. 6 We also used race/ethnicity as an interaction variable to 
identify group differences in the contribution of each predictor variable to the Praxis I test 
performance of each race/ethnicity group. The entry of variables was done in a stepwise fashion 
as presented in the previous descriptive analyses so that race/ethnicity was first, followed in 
succession by UGPA, teacher education program enrollment status, candidate educational 
attainment, candidate’s parental educational attainment, undergraduate major, and selectivity of 
attending institution, and then the interactions of race/ethnicity with each of these variables in a 
similar order, for a total of 13 predictors. Not all predictors were statistically significant for each 
test. The full model results are displayed in Appendix D. 

For ease of interpretation given the number of significant predictor variables, we decided 
that variables explaining at least 1% of the variance in each Praxis I scale score would be 
retained in the model. Race/ethnicity, UGPA, undergraduate major, and the selectivity of the 
attending institution met this criterion in the models for each of the three sets of Praxis I scores. 
Race/ethnicity, UGPA, undergraduate major, and attending institutional selectivity explained 
17% to 22% of the variance in the scores for each of the Praxis I tests. Table 12 displays the 
reduced model results. Standardized regression coefficients are included to allow comparison of 
these results across the Praxis I tests. 
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Table 12 

Summary of Stepwise Regression Results for First-Time Praxis I Test-Takers 



Reading Writing Mathematics 



Predictor 


P 


SE 


Sig. 


Std. p 


2 

r 


P 


SE 


Sig. 


Std. p 


2 

r 


P 


SE 


Sig. 


Std. p 


2 

r 


Race/ 

ethnicity 

UGPA 


-5.47 


.07 


< .01 


-.28 


.10 


-3.21 


.07 


<.01 


-.22 


.05 


-7.15 


.11 


< .01 


-.29 


.11 


2.71 


.06 


< .01 


.20 


.04 


2.17 


.04 


<.01 


.22 


.07 


2.67 


.08 


< .01 


.16 


.03 


Major 


-2.74 


.06 


< .01 


-.20 


.05 


-1.46 


.04 


<.01 


-.15 


.02 


-1.42 


.07 


< .01 


-.09 


.01 


Selectivity 


1.73 


.06 


< .01 


.14 


.02 


1.66 


.04 


< .01 


.18 


.03 


2.37 


.07 


<.01 


.15 


.03 



Note. Please see Appendix C for more information about the terms used in this table. Sig. = significance of regression coefficient, 
Std. P = standardized regression coefficient, UGPA = undergraduate grade point average. 




The analyses indicated that being an African American candidate and being an education 
major were associated with reduced Praxis I scores, while having a relatively high UGPA and 
attending a relatively selective college or university contributed to higher performance on Praxis I 
tests. The regression analyses revealed that White test-takers on average have a three to seven 
point advantage over their African American counterparts on Praxis I. Having a UGPA above 3.0 
as opposed to below 3.0 or having a major other than education gives test-takers about a two to 
three point average increase in scores. Attending a selective college or university on average was 
associated with Praxis I scores that were two to three points higher than those attending a less 
selective college or university. Comparing the standardized beta coefficients across the reduced 
models, race/ethnicity consistently had the strongest effect, between -0.22 for writing and -0.29 for 
mathematics. UGPA had just as strong an effect as race/ethnicity upon writing and explained a 
larger proportion of variance in writing scores (7%) than race/ethnicity (5%). 

The results of the regression models indicated that the significant predictors of Praxis I 
score performance (race/ethnicity, UGPA, selectivity of the attending school, and undergraduate 
major) were all aspects of a candidate’s profile that were either not changeable or not very easily 
changed by the time he or she took the test. A case may be made, though, that undergraduate 
major, which may reflect rigor of preparation, was not fixed since about half of the White Praxis 
I test-taking sample was freshmen or sophomores compared to just under one-third of African 
American candidates. Since African American candidates tended to take Praxis I tests later in 
their academic careers, they may have had a greater likelihood of switching into the education 
domain from their original majors. Learning more about the path to the teaching profession and 
the curricula for African American candidates may provide further insight into changes that 
could be made toward helping to boost scores and reduce gaps. 

The findings confirmed what has been traditionally observed, that the accumulation of 
human capital as represented by various background characteristics is related to higher test 
performance. Of note are results showing the largest racial/ethnic gaps in Praxis I Mathematics 
and how education majors scored lower than their peers who are studying other disciplines. 

We now turn to some of the Praxis II tests and analyses similar to those that we 
conducted on the Praxis I tests to see if similar conclusions can be drawn and to discern whether 
the factors contributing to the performance differences on Praxis II are similar to those found on 
Praxis I. Gitomer, Brown, and Bonett (2008) studied the relationship between Praxis I test-taker 
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performance and Praxis II test-taker performance when taking various background variables into 
account. They found that candidates who struggled with Praxis I were likely to struggle with 
Praxis II as well. 

Findings From Praxis II Analyses 

The 13 high-volume Praxis II tests analyzed in this report represent various content areas, 
such as English, mathematics, and science; and pedagogy, including elementary education and 
principles of learning and teaching. Unlike Praxis I, due to differences in test adoption patterns 
for different tests, states that rely upon Praxis II tests may subscribe to only one Praxis II test or 
as many as needed to serve their purposes. It is therefore important to point out that the 
populations of Praxis I and Praxis II test-takers throughout this report were independent of one 
another, in part due to the short 4-year timeframe employed in the analyses. The degree of 
overlap, defined as those taking all three Praxis I tests and one of the Praxis II tests, ranged from 
0.3% to 10.1%. The issues raised by Gitomer et al. (2008) regarding linking Praxis I 
performance to Praxis II performance cannot be as well addressed in this paper due to the 
constricted timeframe. 

Table 13 summarizes the selected Praxis II tests along with the sample sizes of White and 
African American first-time test-takers. 7 

The range in testing volumes for White test-takers was from 100,622 examinees 
(Elementary Education: Content Knowledge) to 4,174 (Chemistry: Content Knowledge). Among 
African American examinees, the range was from 11,086 (Elementary Education: Content 
Knowledge) to 344 (Chemistry: Content Knowledge). African American test-takers ranged in 
representation from as high as 9.3% (Elementary Education: Content Knowledge) to as low as 
4.7% (Elementary Education: Curriculum, Instruction, and Assessment). White candidates 
ranged in representation from as high as 90.9% (Elementary Education: Curriculum, Instruction, 
and Assessment) to as few as 81.5% of examinees (Elementary Education: Content Area 
Exercises). 

Table 14 compares African American and White test-takers scores on the selected Praxis 
II tests. The African American and White candidate mean score differences varied among the 12 
tests and was appropriately interpreted in the context of the scale score ranges of these tests 
(100-200). Just as with the analyses presented for Praxis I tests, examining these Praxis II data in 
terms of standardized differences rather than scale score differences was preferable. 
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Table 13 



Sample Sizes of Selected Praxis II Tests by Race/Ethnicity Group for First-Time Praxis II 
Test-Takers 





White 


African 

American 


Total 


Praxis II subject test 


Sample 


% 


Sample 


% 


Sample 3 


Elementary education 












Elementary Education: Curriculum, 
Instruction, and Assessment 


70,944 


90.9 


3,700 


4.7 


78,046 


Elementary Education: Content Area 
Exercises 


18,811 


81.5 


2,063 


8.9 


23,076 


Elementary Education: Content 
Knowledge 


100,622 


84.6 


11,086 


9.3 


118,993 


Subject-specific 












English Language, Literature, and 
Composition: Content Knowledge 


34,628 


87.5 


3,226 


8.2 


39,558 


Mathematics: Content Knowledge 


21,440 


83.1 


2,347 


9.1 


25,812 


Middle School Mathematics 


28,059 


86.0 


2,850 


8.7 


32,638 


Social Studies: Content Knowledge 


29,853 


89.2 


2,153 


6.4 


33,451 


Chemistry: Content Knowledge 


4,174 


85.3 


344 


7.0 


4,895 


General Science: Content Knowledge 


5,880 


87.9 


361 


5.4 


6,686 


Middle School Science 


13,119 


89.6 


848 


5.8 


14,635 


Pedagogy 












Principles of Learning & Teaching K-6 


47,182 


88.0 


3,921 


7.3 


53,639 


Principles of Learning & Teaching 7-12 


45,882 


87.5 


3,837 


7.3 


52,461 



a Total comprises the same five race/ethnicity groups displayed in Table 2 (White, African 
American, Hispanic, Asian American, Native American). 



The range of standardized differences was from 0.74 (Elementary Education: Content 
Area Exercises) to 1.41 (English Language, Literature, and Composition: Content Knowledge). 
Other areas with relatively large standardized differences were General Science: Content 
Knowledge (1.37); Elementary Education: Content Knowledge (1.27); Elementary Education: 
Curriculum, Instruction, and Assessment (1.25); and Mathematics: Content Knowledge (1.19). In 
examining the effect sizes, all gaps on the selected Praxis II tests were considered to be large 
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(0.80 and above) with the exception of Elementary Education: Content Area Exercises, which 
was considered to be medium (0.74) yet bordered on being large. 



Table 14 

Selected Praxis II Test Performance by Race/Ethnicity Group With Gaps Presented in 
Standardized Terms 

White African American 



Std. 



Praxis II subject test 


N 


Mean 


SD 


N 


Mean 


SD 


gap 


Elementary education 
















Elementary Education: Curriculum, 
Instruction, and Assessment 


70,944 


177 


13 


3,700 


160 


20 


-1.25 


Elementary Education: Content Area 
Exercises 


18,811 


159 


9 


2,063 


152 


10 


-0.74 


Elementary Education: Content 
Knowledge 


100,622 


166 


15 


11,086 


147 


17 


-1.27 


Subject-specific 
















English Language, Literature, and 
Composition: Content Knowledge 


34,628 


178 


14 


3,226 


158 


17 


-1.41 


Mathematics: Content Knowledge 


21,440 


147 


22 


2,347 


121 


19 


-1.19 


Middle School Mathematics 


28,059 


165 


18 


2,850 


149 


16 


-0.96 


Social Studies: Content Knowledge 


29,853 


168 


15 


2,153 


152 


16 


-1.10 


Chemistry: Content Knowledge 


4,174 


162 


20 


344 


141 


22 


-1.00 


General Science: Content Knowledge 


5,880 


167 


18 


361 


142 


20 


-1.37 


Middle School Science 


13,119 


159 


16 


848 


143 


16 


-0.97 


Pedagogy 
















Principles of Learning & Teaching: 
K-6 


47,182 


175 


11 


3,921 


163 


16 


-1.06 


Principles of Learning & Teaching: 
7-12 


45,882 


174 


10 


3,837 


163 


13 


-1.08 



Note. Std. gap = the standardized gap in mean scores between African American and White 
candidates (AA - W). 



Analyses of selected Praxis II tests in relation to academic major. We further analyzed 
selected Praxis II tests to see if education majors in both race/ethnicity groups performed below 
the levels of their counterparts in other majors, as had been observed for Praxis I tests. This issue, 
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which centers upon both the quality of achievement among people who pursue degrees in teacher 
education and the rigor of teacher preparation program curricula, were part of then current public 
policy discourse revolving around teaching quality. It also raised the question about the 
prevalence of teachers who were assigned to teach in subject areas in which they had not been 
certified. The Education Trust (2008) published a report describing how out-of-field teaching 
persists in key academic courses, particularly in high-poverty and high-minority schools. About 
one in six classes in the core academic subjects of English, mathematics, social studies, and 
science was taught by a teacher not certified in that subject area. When comparing school poverty 
levels, twice as many classes in high-poverty schools (27%) were taught by out-of-field teachers 
as in low-poverty schools (13%; The Education Trust, 2008). 

It was for that reason that certain Praxis II tests were selected for analyses similar to those 
presented earlier for Praxis I tests using background variables. The first set was related to 
elementary education. The second set was related to specific content areas and related concerns 
about out-of-field teaching noted as a focal point in The Education Trust report. 

Elementary education exams with mathematics components. Given that the largest 
observed Praxis I gap between White and African American candidates was on the mathematics 
test, the first set of tests that we selected for analyses were those elementary education exams 
with mathematics components (Curriculum, Instruction, and Assessment [approximately 22% of 
test items] and Content Knowledge [approximately 25%]). Tables 15 and 16 show that the large 
majority of candidates on both tests reported education as their undergraduate major, and this 
group generally underperformed relative to students majoring in other fields. The largest 
race/ethnicity gaps were found among business majors. 

Subject area tests in Mathematics, Social Studies, and English. The next set of tests 
analyzed corresponded to specific subject areas. These included Middle School Mathematics, 
Mathematics: Content Knowledge, Social Studies: Content Knowledge, and English Language, 
Literature, and Composition: Content Knowledge. In order to test the hypothesis that those who 
major in the discipline tend to perform better on Praxis II than those majoring in the field of 
education, we separated undergraduate majors into two groups: test-takers majoring in the 
discipline and their counterparts majoring in education. These findings are presented in the 
following tables and narrative. 
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Table 15 



Elementary Education: Curriculum, Instruction, and Assessment (Elem Ed: CIA) 
Performance by Race/Ethnicity Group and Undergraduate Broad Major Classification Among 
First-Time Test-Takers 



Elem ed: CIA 
undergraduate major 




White 






African American 


Std. 

gap 


N 


% 


Mean 


SD 


N 


% 


Mean 


SD 


Science 


889 


1.5 


181.02 


12.72 


83 


2.7 


161.71 


20.38 


-1.43 


Business 


1,665 


2.7 


178.42 


13.40 


173 


5.6 


157.21 


19.62 


-1.50 


Social science 


4,555 


7.5 


178.72 


13.02 


454 


14.8 


158.03 


22.01 


-1.47 


Education 


51,700 


84.7 


176.06 


13.32 


2,255 


73.3 


160.62 


19.07 


-1.13 


Humanities 


2,211 


3.6 


181.16 


11.85 


112 


3.6 


163.32 


19.78 


-1.44 



Note. Those majoring in technology-related disciplines, those who were undecided, or those 
whose majors did not fit the five major groupings displayed in this table were removed from this 
analysis. Std. gap = the standardized gap in mean scores between African American and White 
candidates (AA - W). 



Table 16 

Elementary Education: Content Knowledge ( Elem Ed: CK) Performance by Race/Ethnicity 
Group and Undergraduate Broad Major Classification Among First-Time Test-Takers 

Elem ed: CK white African American Std 



undergraduate major 


N 


% 


Mean 


SD 


N 


% 


Mean 


SD 


gap 


Science 


2,577 


3.3 


173.72 


16.50 


571 


7.0 


152.69 


17.59 


-1.26 


Business 


3,481 


4.5 


169.08 


15.27 


799 


9.8 


146.45 


16.70 


-1.46 


Social science 


10,373 


13.4 


169.08 


15.75 


1,627 


20.0 


148.51 


17.93 


-1.28 


Education 


55,649 


71.7 


164.87 


15.06 


4,663 


57.2 


145.11 


17.23 


-1.30 


Humanities 


5,576 


7.2 


170.40 


16.19 


487 


6.0 


148.88 


16.77 


-1.33 



Note. Those majoring in technology-related disciplines, those who were undecided, or those 
whose majors did not fit the five major groupings displayed in this table were removed from this 
analysis. Std. gap = the standardized gap in mean scores between African American and White 
candidates (AA - W). 
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Middle School Mathematics and Mathematics: Content Knowledge. Table 17 
displays the performance by race/ethnicity and undergraduate major on the Middle School 
Mathematics test. 

Table 17 



Middle School Mathematics Performance by Race/Ethnicity Group and Undergraduate Broad 
Major Field Classification Among First-Time Test-Takers 



Middle school math 
undergraduate major 




White 






African American 


Std. 

gap 


N 


% 


Mean 


SD 


N 


% 


Mean 


SD 


Science 


2,610 


11.4 


172.48 


17.69 


616 


28.1 


152.50 


17.22 


-1.14 


Business 


2,212 


9.6 


165.12 


16.71 


476 


21.7 


146.63 


14.73 


-1.13 


Social science 


2,403 


10.5 


164.14 


17.74 


266 


12.1 


147.33 


17.05 


-0.95 


Education 


14,831 


64.6 


163.96 


17.01 


764 


34.9 


148.08 


15.24 


-0.94 


Humanities 


915 


4.0 


167.84 


18.47 


69 


3.1 


151.71 


18.42 


-0.87 



Note. Std. gap = the standardized gap in mean scores between African American and White 
candidates (AA - W). 



The majority of candidates were education majors, yet the second largest group was 
science majors. Table 17 also shows that science majors performed better than candidates who 
majored in other fields and that education majors performed generally as well as their counterparts 
in business and social sciences. According to the responses provided in the background 
questionnaire, the major of mathematics is grouped under the larger science domain. As displayed 
in Table 18, only White mathematics majors performed better than mathematics education majors 
on the middle school mathematics Praxis II test. 

The data in Table 18 indicate that the 782 White candidates who were mathematics 
majors accounted for 30% of those classified as science majors among White candidates. 
Comparatively, the 268 African American candidates who were mathematics majors accounted 
for about 44% of those classified as science majors among African American candidates. The 
data in Table 18 also indicate that very few education majors who took this test majored 
specifically in mathematics education. The results on the test show that the gap between 
racial/ethnic groups was slightly smaller among mathematics majors. There were no significant 
differences in performance between mathematics majors and mathematics education majors 

o 

among both White and African American test-takers. 
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Table 18 



Middle School Mathematics Performance by Race/Ethnicity Group for Mathematics 
Education Majors and Mathematics Majors Among First-Time Test-Takers 



Middle school math 




White 






African American 


Std 


refined major 


N 


% a Mean 


SD 


N 


% a 


Mean 


SD gap 


Mathematics education 


715 


4.8 174.04 


16.70 


93 


12.2 


153.23 


14.15 -1.27 


Mathematics 


782 


30.0 174.83 


18.47 


268 


43.5 


152.98 


16.82 -1.21 



Note. Std. gap = the standardized gap in mean scores between African American and White 
candidates (AA - W). 

a The % column is calculated by dividing the sample size for the refined major (e.g., mathematics 
education or mathematics) by the sample size for the higher-order major (education or science) 
displayed in Table 17. 

Tables 19 and 20 show comparable results for the Mathematics: Content Knowledge test, 
which included more science majors than education majors among test-takers, particularly 
among African American candidates. In Table 19, unlike the results shown for the Middle 
School Mathematics test, education majors in both race/ethnicity groups did better than their 
counterparts in business and social science, but still lagged behind science majors. 

Table 19 

Mathematics: Content Knowledge ( CK) Performance by Race/Ethnicity Group and 
Undergraduate Broad Major Field Classification Among First-Time Test-Takers 

White African American 

Mathematics: CK Std. 



undergraduate major 


N 


% 


Mean 


SD 


N 


% 


Mean 


SD 


gap 


Science 


7,561 


44.0 


153.19 


20.79 


955 


52.5 


126.65 


19.38 


-1.29 


Business 


1,269 


7.4 


135.65 


21.30 


303 


16.7 


112.87 


13.93 


-1.13 


Social science 


848 


4.9 


136.71 


21.77 


107 


5.9 


114.94 


13.84 


-1.03 


Education 


7,498 


43.7 


145.02 


21.57 


454 


25.0 


119.57 


17.68 


-1.19 



Note. CK = content knowledge, Std. gap = the standardized gap in mean scores between 
African American and White candidates (AA - W). 
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Table 20 



Mathematics: Content Knowledge (CK) Performance by Race/Ethnicity Group for 
Mathematics Education Majors and Mathematics Majors Who Were First-Time Test-Takers 



Mathematics: CK 






White 






African American 


Std. 


refined major 


N 


% 


Mean 


SD 


N 


% 


Mean 


SD 


gap 


Mathematics education 


5,393 


71.9 


151.25 


18.93 


286 


63.0 


124.45 


18.54 


-1.42 


Mathematics 


4,995 


66.1 


154.32 


20.46 


543 


56.9 


128.29 


19.13 


-1.28 



Note. CK = content knowledge, Std. gap = the standardized gap in mean scores between African 
American and White candidates (AA - W). 



As with the Middle School Mathematics test, the racial/ethnic gap for Mathematics: 
Content Knowledge was only slightly smaller among mathematics majors than mathematics 
education majors, yet it was still large. The results showed slightly higher scores among 
mathematics majors compared to mathematics education majors with small effect sizes (0.16 for 
White candidates, 0.20 for African American candidates). 

Social Studies: Content Knowledge. Social studies may be placed in the domain of 
social science. Table 21 displays the performance by race/ethnicity and undergraduate major on 
the Social Studies: Content Knowledge test. The table shows that the majority of the test-takers 
majored in a social science field compared to education. 

Education majors underperformed on the Social Studies: Content Knowledge test relative 
to their counterparts in social science. The widest gap was also among business majors, who 
comprised a small proportion of the test-takers for this exam (3% of White candidates and 7% of 
African American candidates). Table 22 shows a performance comparison between social studies 
majors and social studies education majors. The frequency of social studies as a major was 
actually quite low and more African American candidates majored in history and political 
science, which are listed as separate categories. 

Test-takers who majored in history or political science performed better than general 
social studies majors. The performance among history majors, in particular, is relevant because 
the gap was the smallest among the listed specializations and was comparable to the gap among 
social studies education majors. 
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Table 21 

Social Studies: Content Knowledge ( CK) Performance by Race/Ethnicity Group and 
Undergraduate Broad Major Field Classification Among First-Time Test-Takers 



Social Studies: CK 




White 






African American 


Std. 


undergraduate major 


N 


% 


Mean 


SD 


N 


% Mean 


SD gap 


Business 


727 


3.1 


171.22 


15.30 


111 


6.8 150.12 17.82 -1.35 


Social science 


13,870 


59.9 


169.52 


14.33 


1,065 


65.3 153.35 16.01 -1.12 


Education 


8,569 


37.0 


165.21 


14.09 


454 


27.9 150.55 14.75 -1.04 


Note. CK = content knowledge, Std. gap 


= the standardized gap in mean scores between African 


American and White candidates (AA - W). 










Table 22 
















Social Studies: Content Knowledge (CK) Performance by Race/Ethnicity Group for Social 
Studies Education Majors and Social Studies , History, and Political Science Majors Among 


First-Time Test-Takers 




















White 






African American 




Social studies: CK 
refined major 
















N 


% 


Mean 


SD 


N 


% Mean 


SD Std ‘ 

gap 


Social studies ed. 


6,701 


78.2 


167.08 


13.16 


292 


64.3 153.48 


14.02 -1.03 


Social studies 


2,128 


15.3 


166.63 


13.79 


121 


11.4 149.91 


14.83 -1.21 


History 


8,963 


64.6 


169.93 


14.14 


540 


50.7 155.18 


16.13 -1.03 


Political science 


1,579 


11.4 


173.29 


13.84 


219 


20.6 155.65 


15.52 -1.26 



Note. CK = content knowledge, Ed. = education, Std. gap = the standardized gap in mean 
scores between African American and White candidates (AA - W). 



English Language, Literature, and Composition: Content Knowledge. English 
was included under humanities. Table 23 displays the performance by race/ethnicity and 
undergraduate major on the English Language, Literature, and Composition: Content 
Knowledge test. 

Even though the gap was smallest among social science majors, as a group, humanities 
majors performed best on this test. Table 24 presents data comparing the performance of English 
education and English majors on the test. 
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Table 23 



English Language, Literature, and Composition: Content Knowledge ( CK) Performance by 
Race/Ethnicity Group and Undergraduate Broad Major Field Classification Among First- 
Time Test-Takers 



English: CK 
undergraduate major 




White 






African American 


- Std. 
gap 


N 


% 


Mean 


SD 


N 


% 


Mean 


SD 


Business 


426 


1.6 


173.58 


15.23 


97 


4.1 


150.10 


16.81 


-1.51 


Social science 


2,226 


8.2 


177.20 


14.23 


264 


11.3 


157.28 


18.33 


-1.35 


Education 


10,159 


37.4 


174.97 


14.24 


708 


30.2 


155.14 


15.81 


-1.38 


Humanities 


14,347 


52.8 


180.58 


13.52 


1,272 


54.3 


160.55 


16.52 


-1.45 



Note. CK = content knowledge, Std. gap = the standardized gap in mean scores between African 
American and White candidates (AA - W). 



Table 24 

English Language, Literature, and Composition: Content Knowledge ( CK) Performance by 
Race/Ethnicity Group for English Education Majors and English Majors Among First-Time 
Test-Takers 

English: CK white African American Std 



refined major 


N 


% 


Mean 


SD 


N 


% 


Mean 


SD 


gap 


English education 


6,748 


66.4 


111. 16 


12.84 


459 


64.8 


158.38 


15.11 


-1.49 


English 


12,718 


88.6 


180.59 


13.49 


1,195 


93.9 


160.56 


16.24 


-1.46 



Note. CK = content knowledge, Std. gap = the standardized gap in mean scores between African 
American and White candidates (AA - W). 



The data suggested that while the gap was approximately equal between English 
education majors and English majors, White test-takers majoring in English performed slightly 
better on this test than English education majors, with an effect size of 0.21, yet the difference 
was not as large for African American test-takers (0.14). 

Finally, while scores tell one story, it was also useful to examine passing rates on these 
content tests based on the specificity of undergraduate majors. Table 25 presents a summary of 
passing rates on the mathematics, social studies, and English tests described in this section. 
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Table 25 

Summary of Passing Rates by Race/Ethnicity Group on Mathematics, Social Studies (SS), and 



English Praxis II Tests Among First-Time Test-Takers 





White 


African American 


_ Pass rate 




Total N 


% pass 


Total N 


% pass 


gap 


Middle school math overall 


28,059 


80.0 


2,850 


48.2 


-31.8 


MS math - math ed. majors 


715 


90.9 


93 


59.1 


-31.8 


MS math - math majors 


782 


89.3 


268 


59.0 


-30.3 


Mathematics: CK overall 


21,440 


72.6 


2,347 


25.3 


-47.3 


Math CK - math ed. majors 


5,393 


83.1 


286 


34.6 


-48.5 


Math CK - math majors 


4,995 


82.1 


543 


36.1 


-46.0 


Social studies: CK overall 


29,853 


83.9 


2,153 


47.6 


-36.3 


SS CK - social studies ed. majors 


6,701 


86.3 


292 


54.5 


-31.8 


SS CK - social studies majors 


2,128 


81.7 


121 


45.5 


-36.2 


SS CK - history majors 


8,963 


85.3 


540 


52.8 


-32.5 


SS CK - political science majors 


1,579 


90.7 


219 


56.6 


-34.1 


English: CK overall 


34,628 


88.7 


3,226 


46.4 


-42.3 


English - English ed. majors 


6,748 


91.4 


459 


51.6 


-39.8 


English - English majors 


12,718 


90.7 


1,195 


51.0 


-39.7 



Note. CK = content knowledge, MS = middle school, SS = social studies, Pass rate gap = the gap 



in passing rates between African American and White candidates (AA - W). 

The analyses presented in Table 25 suggest that majoring in a discipline/content area 
instead of education within that same discipline/content area was not related to higher passing 
rates for candidates in general, particularly for Social Studies: Content Knowledge. The one 
exception was for African American candidates on Mathematics: Content Knowledge where the 
passing rate was higher for mathematics majors compared to mathematics education majors. 
These findings did not appear to confirm the added value of undergraduate major field of study 
in explaining Praxis II performance, which had been established earlier in the section of Praxis I 
performance. 

It is also worth mentioning that the range of gaps in passing rates on these Praxis II tests 
was similar overall to those found for Praxis I tests. However, the gaps in passing rates on the 
Mathematics: Content Knowledge test were larger than the mathematics gap for Praxis I. 
Because of the limited overlap between our samples of test-takers taking Praxis I and Praxis II, 
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the analyses were limited. But, we hypothesize that the apparent mathematical-related 
differences detected on Praxis I Mathematics carry through into Praxis II mathematics tests. 
These differences are likely related to academic factors beyond those captured in the BIQ like 
student preparation and curricula and may also be related to nonacademic factors, such as 
anxiety toward the subject which has been shown to intensify with age (Hembree, 1990). 

The results for the Praxis II mathematics tests were also interesting because of three 
points noted by The Education Trust (2008) in relation to teacher knowledge: (a) there seems to 
be a strong link between teachers’ knowledge of mathematics and student achievement; (b) 
student success in mathematics is the most reliable predictor of success in college and the 
workplace; and (c) national and international data indicate that many U.S. students lack the 
mathematics knowledge and skills to compete in the global economy. 

In order to see if the correlates identified in analyses of Praxis I tests applied to the Praxis 
II tests selected in this section, we performed regression analyses similar to those that we 
conducted on Praxis I tests. The scale score for the selected Praxis II test was the dependent 
variable, and the background variables of race/ethnicity, UGPA, major, and attending 
institutional selectivity were the independent variables. The regression analyses are presented in 
Appendix D. 

The results were similar to those found for Praxis I in that race/ethnicity explained the 
most variance in the scale scores of the selected Praxis II tests, and in a similar range of 4% to 
10%. UGPA explained slightly less variance in Praxis II scores (1% to 3%) than it did for Praxis 
I (3% to 7%). The candidate’s undergraduate major and selectivity of his or her attending 
institution explained comparable proportions of the variance in scores as for Praxis I. Since the 
scales of Praxis II scores are quite different (100 to 200) than the Praxis I score scales (150 to 
190), the standardized regression coefficients are most relevant in comparing these model results 
to those previously shown in Table 12. The results showed that as with Praxis I, race/ethnicity 
had a similar effect on Praxis II scores as on Praxis I scores, namely that scores of African 
American candidates were likely to be lower than those of White candidates. While the signs of 
the standardized regression coefficients were the same for UGPA, major, and attending 
institutional selectivity, the magnitudes of these effects were generally smaller than those for 
Praxis I. While the results in the beginning of this section indicated that major field of study was 
a significant factor influencing Praxis II performance, the regression results showed that it was 
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less of a factor when students attended similarly selective colleges and universities and were of 
the same race/ethnicity. 

The quantitative analyses in this report have focused on race/ethnicity group differences 
(primarily between African American and White prospective teachers) in their performance on 
high- volume Praxis I and Praxis II tests. Using descriptive statistics (e.g., mean scores and effect 
sizes) and regression analyses, performance differences between African American and White 
test-takers were within expectations based on various demographic indicators and other 
characteristics beyond simply examining race/ethnicity. These differences tended to be similar 
across the variety of Praxis tests. Other sources for the differences were beyond the scope of 
available background data. For that reason, we also conducted a qualitative analysis. 

The next section of the report presents what a sample of minority-serving institution 
faculty considers to be some of the challenges and their efforts to help remedy the problem of 
under-representation of minority teachers among those holding or achieving certification. 

On-Site Campus Interviews 

The qualitative component of the research involved visits to seven campuses to interview 
teacher education faculty and students. Staff from ETS and NEA conducted the interviews. 

While exploratory in nature, the purpose was to gain firsthand knowledge from the faculty about 
the challenges that their students confronted when taking licensure tests and to elicit their ideas 
for improving student performance. ETS periodically hears unsolicited concerns from Praxis 
test-takers about their testing experiences, but this was the first time an in-depth exploration was 
conducted to explore issues with those who prepare teacher candidates. 

The campus-based interviews engaged faculty in structured conversations regarding their 
concerns and activities related to preparing their students to become licensed teachers. In 
addition to specific questions regarding preparation for Praxis, faculty views about the benefits 
of the licensure process for higher education and the teaching profession were also elicited. 
Professors described the ways the licensure testing process affected their course syllabi and 
curricula design. The interviewers were also interested to hear from students about their 
experiences with Praxis testing. These interviews yielded information about their preparation and 
testing processes. 

Teacher candidates arrive at higher education with varying backgrounds, levels, and 
types of preparation. Among the many factors that are typically found to be related to their 



36 




educational achievement and performance are the curricula and opportunities that they 
experienced in high school, their college grades, their parents’ educational levels, and their social 
class (Nettles, Millett, & Oh, 2009). Faculty of teacher preparation programs, as in other 
institutions, must accommodate the variety of backgrounds, preparation, and achievements that 
students bring to their classrooms and work toward equipping their students to pass the teacher 
licensure tests and to excel as teachers. These circumstances suggest that a variety of teaching 
strategies may be required. The interviews were valuable for the specific information revealed on 
how students were being prepared to succeed on their teacher licensing exams and in their 
teacher education programs, and also for identifying needs and opportunities for the concerted 
efforts of colleges and universities to help prepare students. Much of the interviews centered on 
the efforts to align campus coursework, workshops, and special study sessions with the content 
of the Praxis exams. 

Selection Process 

In order to focus attention on African American, Hispanic, and Native American 
candidates who are likely to experience the greatest challenges in their teacher education 
programs, we sought to identify a small sample of higher education institutions that mainly serve 
these populations. We included three Historically Black Colleges and Universities (HBCUs), two 
Hispanic-serving Institutions (HSIs), one majority institution with a large Native American 
enrollment, and one urban majority institution with a diverse minority population. 

The criteria for selecting the seven institutions included the following: 

• Accreditation by the National Council for the Accreditation of Teacher Education 
(NCATE) 

• Location in a state that uses Praxis tests for licensure 

• Graduation of at least 40 teacher candidates each year 

• Presence of an active teacher education program 

The rationale for selecting the programs based on the number of graduates was that such 
institutions, by virtue of the presence of the academic field of teacher education, demonstrate the 
commitment to preparing teachers. Even though institutions without teacher education programs 
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per se often engage in preparing teachers, the institutions that explicitly offer teacher preparation 
programs often give priority status to educating and preparing future teachers. 

Geographic distribution and public-private representation were also important 
considerations. Satisfying the former objective was somewhat challenging because Praxis is not 
used in several southwestern states or in some of the most densely populated states, such as 
California, New York, and Illinois, where the Hispanic population is large. Efforts to include 
private minority-serving institutions were ultimately unsuccessful because most of them are 
small and do not graduate 40 teacher candidates per year, and many of them do not have teacher 
education programs. 

By examining campus websites for the breadth of teacher education programs and by 
reviewing the number of Praxis test-takers who sent their score reports to their home institutions, 
we identified seven universities that met our criteria, using an initial 1-year extract of Praxis 
candidates from the larger sample used throughout this report. The teacher preparation programs 
of the seven colleges and universities show a large variation in demographic diversity among 
enrollments. 

As a group, these institutions are all major producers of teachers for their respective 
regions, whether these are large urban school districts or within the state as a whole. Four of the 
seven HBCUs and other minority serving institutions that participated in this NEA-ETS 
investigation ranked among the nation’s top 50 producers of minority graduates with 
baccalaureate degrees in education. 

Planning and Implementing the Campus Visits 

For each campus selected, an NEA or ETS interviewer contacted the dean of the school 
of education by telephone, describing the project and seeking her or his desire to participate. (See 
Appendix E for the telephone script inviting campuses to participate). Follow-up e-mails 
confirmed the visiting team, possible dates, agendas, and desired campus participants for the site 
visits. While the goal was to have more or less uniform visits, this was not always possible given 
academic schedules, faculty and administrators’ responsibilities, and student availability. 

The interview teams conducted 1-day site visits of 3 to 5 hours at each of the seven 
campuses during April and May 2006. The composition of the research teams varied by site, but 
for most visits, both ETS and NEA were represented. The teams interviewed more than 50 
faculty and administrators, including deans of education, deans of arts and sciences, chairs of 
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elementary education, secondary education, curriculum and instruction, assessment coordinators, 
directors of math clinics, directors of undergraduate studies, directors of licensure, and faculty 
from a number of disciplines. On some campuses, and when the institutions suggested, the teams 
also interviewed students. 

Using an interview guide (see Appendix F) as a starting point, each interviewing team 
found that the proposed topics led to far-ranging discussions regarding the challenges faced by 
minority students, the amount of commitment displayed by students to careers in teaching, and 
the degree to which faculty and administrators were willing to shape their curriculum to support 
student success in the quest for licensure. We learned of a variety of approaches including 
required semester-long Praxis preparation courses, optional weekend workshops, and making test 
preparation materials available, both in print and electronically. In many of the institutions, 
faculty were encouraged (sometimes required) to take appropriate Praxis licensure tests 
themselves in order to gain more knowledge about the tests and in turn better prepare their 
students to take the tests themselves. The colleges and universities sometimes allocated funds to 
faculty to pay the fees required to take the tests. In some cases, arts and sciences colleagues also 
took the tests (e.g., for mathematics and English) since these faculty played a major role in 
preparing teacher candidates in these fields. Several teacher education faculty described 
incorporating some of the Praxis practice materials directly into their courses in order to increase 
the degree of student comfort. Details about what was learned from the site visit interviews 
follow here. 

Observations and Findings: What We Learned From the Campus Interviews 

Keeping in mind the context provided by the quantitative data, we turn now to the results 
of the qualitative analysis. In our examination of campus challenges and perceptions, we sought 
to explore the following: 

• University approaches to address the need for Praxis preparation 

• Common campus-based misconceptions about Praxis tests 

• The major hurdles faced in efforts to close achievement gaps on Praxis tests 

Since this field work design was nonrepresentative and nonrandom, we were careful about the 
extent to which we try to generalize conclusions from the information gained during the 
interviews. The design does, however, represent a useful way to converse with college and 
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university faculty and administrators about the challenges faced by their students. ETS will build 
on these interviews and consider ways to expand and improve the dialogue. 

Praxis-specific preparation within/outside the curriculum. No uniform Praxis 
preparation program was observed across the campuses visited. Each institution developed its 
own preparation program in accordance with its own traditions, perspectives, and ideas, and its 
views of its students’ needs for support. The colleges and universities visited in this inquiry, 
however, used two basic models for Praxis support. The first was to integrate Praxis content 
directly into their credit coursework. For example, one campus described how professors in 
science education required students to practice constructing and responding to Praxis-like 
questions in science education courses. This model seemed more prevalent for Praxis II support 
but was also observed for Praxis I. 

The second model was to provide free-standing Praxis preparation, either in the form of a 
test preparation course or Praxis clinics. This type of Praxis support varied from a 
comprehensive approach, as in the test preparation course, to a more independent study 
arrangement. In the latter cases, diagnostic tools were used to identify students who were then 
referred to existing labs or clinics to address particular weaknesses. The free-standing Praxis 
model incorporated both diagnostic and remedial components. Institutions also used the PLATO 
Simulated Pre-Professional Skills Test, created by PLATO Learning to identify areas of 
weakness, and then provided clinics to address those weaknesses. The faculty reported the 
frequent use of counseling to address test anxiety issues, and the faculty at one school noted that 
this anxiety seemed to be more prevalent among African American students. 

Although the PLATO program seemed to be the preferred tool used for Praxis support, 
faculty at several colleges and universities discussed its inadequacies. The most common 
complaint was the lack of test preparation questions. This issue was raised both about PLATO 
and in reference to ETS. Faculty at one university reported offering to work with the company 
that marketed PLATO as a way of helping shape the program’s responsiveness to its students’ 
needs, as well as to identify “bugs” in the program. 

Using Praxis study materials and programs. The primary resources used across the 
seven campuses to help students prepare for Praxis are PLATO (a Praxis I online tutorial), ETS 
Tests at a Glance (TAAGs), 9 free downloadable test descriptions, and for-purchase ETS Study 
Guides and retired tests. TAAGs were deemed to be useful for reading and essays in Praxis I. 
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Textbooks were thought to be more useful for mathematics. ETS Study Guides for Praxis II were 
not seen as particularly useful, and one interviewee commented that the length of reading 
passages in the Study Guide books did not match the lengths of the reading passages on the 
actual tests. This was a concern because students often complain about the length of the 
passages, and faculty/administrators pointed out that practice was one of their methods of 
making the students more comfortable. According to our interviewees, this lack of comparability 
between the tests and preparation books limited the usefulness of this practice tool. 

While several of the colleges and universities reported circulating the ETS Institutional 
Summary Reports (ISR), they found them not specific enough to be really useful. This, plus the 
length of time between administering the tests and receiving score reports combined to make it 
difficult for colleges and universities to easily diagnose their students’ problems. Faculty 
lamented what they saw as a shortage of released items and indicated that having more released 
items would allow them to work better with their students, diagnose problems, and provide 
needed remediation. 

Supplemental courses, workshops, and clinics. Four of the seven campuses (including 
all three of the HBCUs) required a Praxis I introductory course for admission to the teacher 
education program. In most cases, this was a semester-long course that covered reading, writing, 
and mathematics. Instructors used practice questions and tests to diagnose weaknesses and 
design interventions for improvement. On one campus, at the end of the introductory course, 
students took the Praxis I exam together, as a cohort. The university funded their first attempt at 
Praxis I, so that in addition to the natural desire to do well on the test, there was a financial 
incentive. 

Tutors and math and writing clinics were available on most of the campuses. In addition, 
some institutions provided Praxis-specific tutorials, including Saturday workshops. Students 
reported that study groups helped them, as did workshops about how to read the questions and 
about time management. One institution described its collaboration with a local magnet school as 
a way to encourage future teachers. To help those who were then teaching in neighboring areas, 
but not yet permanently certified, one campus provided yearly workshops to local schoolteachers 
on the Praxis tests required in that state. Over a 10-month period, faculty worked with the school 
district’s television station to develop a series of videotapes in which faculty from the various 
academic disciplines discussed the Praxis II subject tests and reviewed information that was 
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provided in the TAAGs. They also talked about what materials and study guides were available 
to prepare or to gain subject matter proficiency necessary for successful performance on the 
tests. 

Faculty. The degree to which teacher education faculty had direct experience with the 
Praxis tests varied widely. One college/university actively urged its faculty to take the tests by 
setting aside funds to support the activity. Others merely encouraged it, with varying degrees of 
success. Faculty members who had taken the tests reported that they gained a better 
understanding of what was expected of their students; several of them also commented that they 
had incorporated Praxis-like items into their own classroom tests as a way to prepare students. 
On one campus, the faculty interviewed objected to the notion of any direct test preparation in 
their classrooms; they instead chose a stand-alone course to improve the preparation of their 
teacher candidates. 

Close cooperation between arts and sciences (A&S) faculty and teacher education faculty 
was universally recognized as a goal for teacher education programs, particularly in the 
secondary certification disciplines. Most schools reported some degree of cooperation, but it was 
uneven, with some A&S departments actively participating and others doing so less 
enthusiastically. The nature of the cooperation ranged from, for example, one joint faculty 
meeting per year to a Praxis coordinating group across the academic divisions. Some faculty 
reported working together with A&S colleagues on grants and team-teaching courses. When 
significant cooperation with A&S was achieved, the faculty at the schools of education reported 
improvements and increased success in raising passing rates. 

As part of the site visit discussions, faculty were asked if they had ever written test items 
or served on test development committees for ETS. Very few had participated in these activities, 
but many indicated that they would be interested in serving on test development committees and 
receiving training in item development. They were invited to send their curriculum vitae and 
contact information to the ETS visiting team member and were promised that additional 
information would be forthcoming. 1(1 The interviewees encouraged ETS to reach out more 
proactively to education faculty, especially those serving in schools with a significant minority 
enrollment, to work with them on developing and reviewing Praxis tests. Similarly, very few 
faculty had ever participated at the state level in content review panels for the licensure tests. 
This was seen as an issue needing further attention. 
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Consistently, faculty saw Praxis I as a barrier to recruiting and retaining minority teacher 
candidates. They believed this was especially problematic given regional and national teaching 
shortages and the under-representation of minority teachers within an increasingly diverse K-12 
student population. The administrators and faculty at the institutions visited cited perceived test 
bias and inadequate preparation in math and reading comprehension in rural and urban K-12 
schools as causes for lower Praxis I pass rates on the part of minority and nontraditional teacher 
candidates. Analyses of the testing results described earlier in this report shed more light on a 
number of factors that influence the performance of minority candidates on Praxis I. 

Faculty of color who were interviewed believed that an increase in test items with 
relevance to the minority experience could help to close the gaps. They believed that this 
increase could be accomplished through greater involvement of minority educators in the test 
development process, both as item writers and reviewers. 

Students. On most campuses, we interviewed faculty about their students, as well as 
some students themselves. The responses that follow are grouped according to their source. 

Faculty on their students. Chief among the comments from faculty were those that 
focused on students’ high school academic deficiencies and students’ lack of motivation to 
prepare for Praxis tests. All mentioned a serious challenge to all efforts at providing Praxis 
support: how to make sure that minority students take full advantage of services offered. In spite 
of the fact that many minority students arrived with educational deficiencies, they often had to be 
pushed to go to the labs and to devote the necessary time to the work required to pass the tests. 
These students tended to cite more pressing priorities as an explanation for not pursuing the work 
in the Praxis courses and labs. The faculty and administrators associated this with attitudinal 
issues, but also with demographic differences, because these students tended to be younger and 
with fewer financial resources — and therefore less free study time — than the students of other 
racial/ethnic groups. 

Some faculty reported employing a variety of motivational tools to encourage 
participation, and all of them reported doing lots of “hand holding” and encouragement to help 
their students “get over the hump.” A particularly compelling model on one campus was that of a 
full-time staff member dedicated to supporting and encouraging teacher education students. 

Specific areas of deficiency 11 were identified as basic mathematics (especially geometry, 
fractions, decimals, and percentages, and word problems in particular), reading comprehension 



43 




and processing information, application of knowledge to a problem, time management, and 
attention to detail. Word usage and idioms were identified as specific areas of concern in writing, 
as was the length of the reading passages. Vocabulary was an issue for English-language learners 
(ELLs). In general, faculty felt that students should read more. 

Students on themselves. In addition to customer service complaints, access issues, and 
cost challenges, students offered the following comments about their performance and 
achievement on Praxis: 

• Time limits are a problem for all, but especially for nonnative speakers of English, 
many of whom seemed to be unaware of the availability of accommodations through 
the Primary Language Not English program that allows for up to 50% additional 
testing time. 

• Short readings that often contain vocabulary /idioms unfamiliar to those who have not 
been raised in the United States are viewed as problematic. Some words are seen as 
rare and archaic, and some topics are unfamiliar to minority students. 

• Some older, career changers who did not attend K-12 schools in the United States 
lack background in U.S. history and cultural history. They thought questions on these 
topics on Praxis I should be avoided. 

• Online essay practice is a big help, as are face-to-face study groups. 

• It is difficult to pull out the actual arithmetic problem from word problems. 

• More training in test-taking strategies is needed. 

Findings gleaned from our conversations with faculty during our campus visits, as well as 
the testing results discussed earlier, pointed to the need for interventions to compensate for 
academic deficiencies resulting from poor educational preparation. The reported lack of 
alignment and articulation with area high schools seemed to be a particularly disturbing deficit. 
Since Praxis I concentrates on content learned in high school, or so-called basic skills, 
strengthening the connections between high school and teacher education programs at colleges 
and universities may have a significant impact on minority passing rates. This strategy deserves 
attention, particularly since all of the schools believed that Praxis I had more of a negative 
impact on the minority teacher pipeline than Praxis II. 
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State Policies Pertaining to Praxis Tests 

Faculty on nearly all of the campuses visited mentioned that increased emphasis on their 
Praxis programming efforts was the result of changes in their respective state’s certification 
policies. These included increases in the passing score or requirements to use Praxis as a 
gatekeeper for licensure. In spite of the schools’ efforts to change their programs in response to 
the policy changes, many of the faculty and administrators interviewed indicated concern that 
changes in state policies with regard to Praxis tests had the danger of further decreasing an 
already small pool of minority teachers, as well as increasing the percentage of minorities who 
entered through alternative certification routes, where Praxis I is not required. Many complained 
that their respective state’s policies were developed without their input. 

Many faculty were not aware that each state uses a different array of Praxis tests and 
passing scores. Each state’s Board of Education or the Professional Standards Board selects the 
tests that will be required for licensure and then sets passing scores on these tests through an 
extensive standard setting-study process that involves representative groups of faculty and 
practitioners who make judgments about the test items and forms. Neighboring states may — and 
often do — require different tests. Because of different passing scores in different states and 
limited reciprocity, a student who passes the test in one state could be considered failing in a 
different state. The variation in passing scores gives rise to questions regarding the relationship 
of a passing score to the quality and effectiveness of teachers who pass the test. 

Conclusion 

While the potential benefit of combining quantitative and qualitative research and 
analyses in a single study to address critical policy issues is often acknowledged, rare is the 
occasion to read such an endeavor in the literature. This study combines the two approaches to 
address the critical test score gaps between majority and minority test-takers on Praxis. The 
research sought evidence to support actions that can be taken to reduce the gaps. 

The score gaps in all educational assessments seem acute and Praxis is no exception. 
Given the racial/ethnic imbalance in representation in the teaching profession, the challenge is 
especially troubling for teacher licensure tests. Because Praxis is the licensing test of choice for 
the majority of states, it is particularly useful and timely to examine the larger issue of score gaps 
and passing rates by making use of the teacher licensure tests most frequently selected by the 
states. 
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At around 8% of the total teaching workforce in the United States, African American 
teachers are severely under-represented relative to the 13% that they comprise in the adult 
population. Their representation among the teaching workforce is half of their representation 
among the school-age population (Planty et al., 2008, p. 85, Table 5.1). While alternatives to the 
conventional routes to the teaching profession are growing, the primary source of new teachers 
continues to be the colleges and universities that award bachelor’s degrees in the field of 
education. As long as the proportion of bachelor’s degree recipients in the field of education 
continues so that African American awardees represent 6% of all awardees, their under- 
representation in the nation’s teaching workforce will continue to be below the 13% that they 
represent in the adult population. 

Trends in the teaching workforce suggest that a growing share of new teachers is 
emerging through alternative routes. Even though it is clear that the number of alternative routes 
as well as the number of people in them have been increasing in recent years and that they are 
contributing more to the overall supply of new teachers, it is difficult to judge their contributions 
to addressing the shortage of African American teachers for the following reasons: 

• While gaining in popularity, these alternate route programs combined produce 
roughly one third as many new teachers as conventional teacher education programs 
but it is not known what percentage of new African American teachers are being 
produced through alternative routes. 

• The longevity of alternative route teachers as practicing teachers relative to their 
contemporaries is not known either overall or among African Americans. 

• Surveys of alternative route teachers tend to report the proportion that minorities 
represent, but they disaggregate minority alternative route teachers sufficiently to 
distinguish African American alternative route teachers from those from other 
minority groups. 

Regardless of the education or training path people take on the way to becoming a 
teacher, Praxis and other licensure tests and the standards of quality that are represented by each 
state’s passing scores also play a prominent role in addressing the problem of the under- 
representation of African American teachers. African American test-takers comprised 5% to 11% 
of the Praxis volumes in the 28 states during the period covered in the analyses presented in this 
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report, and they represented roughly between 5% and 13% of the test-takers who met the test 
score required to be licensed across the Praxis tests covered in this report. 

Given the currently limited supply of African American teachers, any leak in the pipeline 
is problematic. It is also problematic because, like people who pursue alternative routes, the 
people who take Praxis and other licensing tests represent the only known supply pool that is 
truly interested in becoming teachers. Even baccalaureate degree recipients in the field of 
education cannot be considered interested and committed to becoming teachers until they register 
to take required licensing tests. 

The analyses in this report revealed very large score gaps between African American 
and White teacher candidates on selected Praxis I and selected Praxis II tests. The overall 
gaps, however, between African American and White test-takers on Praxis I appeared to be as 
large as the gaps that are commonly observed on the SAT and GRE. The Praxis I Writing gap 
appeared to be somewhat narrower than gaps on the SAT and GRE, the reading gap was 
comparable, and the mathematics gap seemed larger. Rather wide gaps also existed on the 
majority of the selected Praxis II tests as well. 

Teacher licensing test scores and the passing rates are but two indicators of African 
American Praxis test-takers’ under-achievement that need to be addressed. The data presented in 
this report revealed lower UGPAs of African American test-takers that mirror the differences 
observed in their Praxis I scores. The relationship between undergraduate grades and Praxis test 
scores suggested that the supply of new African American teachers obtaining licenses would be 
similar regardless of whether states used GPAs or Praxis test scores as the primary criterion for 
licensure. If grade inflation were not a likely consequence, and if GPAs could be standardized 
across disciplines and institutions, states might be tempted to try relying upon grades to license 
candidates. Employing Praxis as a licensure criterion relieves the burden that GPAs and the 
faculty who award them would have to bear. 

The strong relationship that we observed between Praxis I scores and UGPA suggested 
that the key to increasing the supply of African American teachers among those who are 
interested in pursuing a license by taking the tests is to focus upon strengthening their academic 
preparation for and achievement in college. Improving grades and school performance are 
important, yet these are not the most important factors. In fact, the finding in this study, that as 
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African American and White test-takers’ grades increase so too did the gaps on Praxis I, 
suggested that other factors are at work as well. 

This phenomenon of test score gaps widening among candidates with higher grades may 
be due in large part to the higher rate of African American candidates attending and earning their 
grades in less selective/competitive colleges and universities than their White contemporaries. 
This widening gap may also be a function of the economic status and standing of individual test- 
takers, which may also play a part in their preparation for higher education and the selectivity of 
the colleges and universities they attend. Preparing for and attending a selective college or 
university may be outside the financial reach of too many prospective teachers and outside of the 
reach of more African American candidates than White candidates. While preparing to attend 
more selective institutions may not be an option or even an interest to many students who are 
preparing to become teachers, it is an issue that should be on the minds of faculty and 
policymakers as a matter to consider in their efforts to close gaps in performance. Interestingly, 
although having a larger proportion of African Americans attend more selective colleges and 
universities may contribute to narrowing the test score gap, given that gaps were larger among 
students attending more selective institutions, this is unlikely to solve the score gap problem. 

Although the relationships between test-taker characteristics and performance on each of 
the three Praxis I tests varied, it is noteworthy that the relationships of SES, like selectivity of 
undergraduate institutions, and UGPA were consistently found to be prominent factors 
associated with Praxis I scores of African American candidates. While race/ethnicity and being a 
noneducation major were found to be highly relevant in predicting all three Praxis I scores, this 
combination was especially prominent in predicting mathematics scores. Given that the Praxis I 
score gaps in mathematics were the widest and were wider than gaps on other tests like the SAT 
and GRE, it is imperative that we examine further why, after controlling for the other 
background characteristics, being an African American prospective teacher is still a significant 
factor. This examination meant that even when African American test-takers attended similarly 
selective colleges and universities, had a parent with comparable educational attainment, 
achieved similar undergraduate grade point averages, were in similar major fields, and were at 
comparable stages of the educational process as their White counterparts, they were still likely to 
have a score seven points lower on the Praxis I Mathematics test compared to their White 
contemporaries. 
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Learning that a substantial share of the differences in mathematics scores on Praxis I can 
be attributed to race/ethnicity was quite illuminating. That information means that perhaps 
efforts to understand the gap in mathematics, in particular, require a more refined and nuanced 
examination of variables within the African American test-taking population than are available in 
the BIQ. Variables that could be considered include the following: the number of mathematics 
courses taken in high school and college as well as the content and quality of those courses, 
teachers, and faculty, African American students’ orientation to mathematics prior to college, 
and the GPAs earned in mathematics courses during high school and college. The same strategy 
should be followed for examining the effects of race/ethnicity upon the score gaps in writing and 
reading. The BIQ variables, while useful for gaining general insight, are not precise enough and 
are insufficiently elaborate to explain all the differences in scores. 

The differences between African American and White test-takers go beyond their scores, 
and these differences need closer scmtiny for their possible connection to the score gaps. African 
American Praxis I test-takers in this study were older on average, took the test at a later stage in 
their academic careers, and were more often already bachelor’s degree recipients. African 
American test-takers were also from lower SES backgrounds and were less likely to be attending 
a selective college or university than their White counterparts. Gitomer et al. (2008) posed the 
question of whether Praxis I was an obstacle to pursuing a teaching career or was actually 
measuring content and skills essential for later success. Their analyses used 3 years of Praxis I 
and Praxis II performance data to focus on the ability of candidates to pass selected required 
Praxis II licensure tests, namely English, mathematics, social studies, and elementary education, 
after taking Praxis I. Candidates were classified into two groups: Successful (meeting or 
surpassing the median passing score used across states) or Borderline (not exceeding the median 
passing score or meeting it only after taking one or more of the Praxis I tests multiple times). 

The findings indicated that passing Praxis I successfully on the first attempt increased the 
chances of passing Praxis II compared to those who struggled to pass Praxis I. These results were 
reported for all test-takers and by subgroups. African American test-takers who successfully 
passed Praxis I the first time were just as likely to pass their Praxis II exams as White test-takers 
with similar success on Praxis I and similar undergraduate GPAs. These findings suggest that 
when African American students enter their teacher education programs with a skill base from 
pre-kindergarten through Grade 12 (P-12) and do not struggle to pass Praxis I tests, the 
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difference around program success and Praxis II performance virtually disappears. For those in 
the Borderline group, however, struggles with Praxis I are indicative of later difficulty, namely 
with Praxis II tests. 

Anticipating the limitations of the BIQ variables for explaining the gaps and considering 
possible strategies for improvement gave added value to the qualitative component of this study. 
In addition to gaining an understanding of the views of faculty and students in some leading 
institutions that are also among the leaders in training new teachers, the qualitative study also 
helped to identify new variables for potential use in the BIQ and elsewhere for attempting to 
explain and address the score gaps between African American and White test-takers. This study 
represented a novel approach that can be employed by researchers who are interested in the 
greater understanding that might be gleaned from conversations with faculty and students. 

While many efforts are being made by organizations and by teacher education faculty to 
help minority candidates succeed on their licensure exams, it is clear from interviews with 
faculty and students that more needs to be learned and more action taken. Based on information 
gathered for this study from faculty, students, and expert observers, many aspects of preparation 
could be improved. These improvements in preparation should result from the following: 

• Better alignment of high school curricula with Praxis I skills 

• Helping teacher candidates understand that the skills and knowledge assessed on 
Praxis tests are important for teaching 

• Helping teacher candidates build skills and knowledge, understand how they will be 
tested, and understand how they can use strategies for successful demonstration of 
their abilities in a demanding testing situation 

• Helping teacher candidates understand what is on each test, how they can prepare, 
and where they may go to seek helpful and authentic practice material 

Many students need guidance through the materials and study processes, either in groups or in 
one-on-one settings. It is clear that the commitment from some faculty is high and that they 
believe an array of materials can help. 

As is true of most studies about teacher supply, attrition is often the result of students’ or 
practicing teachers’ desires to switch fields or careers to seek higher salaries, or family-related 
issues that are independent of licensure assessments. Torres, Santos, Peck, and Cortes (2004), 
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through The Education Alliance at Brown University, reported on aspects of recruitment, 
development, and retention among minority teachers. Among the issues related to retention even 
before these candidates enter classrooms are the lack of acknowledgement of these candidates’ 
diverse backgrounds within preparatory programs. The focus would then seem to shift back to 
the P-12 system to identify the weak links in the chain. As part of the recruitment process, 
Haselkorn (2000) suggested the formation of clubs, programs to build up career awareness, and 
visits by high school students to college campuses that have teacher preparation programs. 
Partnerships between high schools and colleges and universities can also assist in this regard. 
Banks (2002) recommended that community colleges in particular be viewed as sources of 
untapped potential to increase the pool of prospective teachers. 

Ingersoll (1999) recommended individualized diagnostic assessments to help faculty 
select appropriate courses and determine the amount of necessary academic support for students, 
to provide tutoring and focused advisement, and to monitor progress to help improve their 
academic experiences. Once in classrooms, The Education Alliance report (Torres et al., 2004) 
suggested mentoring as a key factor in retention to help promote individual teacher self-efficacy, 
so as to not just retain minority teachers in the profession, but to reduce the amount of transfer 
(Smith & Ingersoll, 2003). 

The evidence is robust that the supply of minority teachers is affected by more than their 
success on licensure exams. Other questions also need to be continuously addressed including 
the following: Do similar differences exist related to other requirements such as program 
completion, GPA, coursework, and student teaching requirements? Are a disproportionate 
number of minority teachers leaving the classroom once they are in the profession? The chain 
can be broken in numerous places, and while the observations reported in this report on Praxis 
performance differences are just one clue to answering these questions, it is important to 
understand the other links and weaknesses in the supply chain as well. 
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Notes 



1 Praxis I tests given by computer were not considered for the analyses in this paper. 

2 Please refer to http://www.ets.org/Media/Tests/PRAXIS/pdf/01361reg.pdf for a list of 

questions. 

Exceptions are states using composite scoring methodologies where no minimums are 
necessary on any test as long as the composite score for all three tests is met. 

4 The inclusion of information about standard errors (i.e. the ratio of the standard deviation to the 

square root of the sample size) can also be useful in interpreting descriptive data. This 
information is available from the authors upon request. 

5 Please see Appendix C for more details on the use and interpretation of this statistic. 

6 Please see Appendix B for how the BIQ variables were consolidated for the regression analyses 

in this paper. 

' 7 

The Early Childhood Education title was dropped from analysis because the score range (200- 
800) was vastly different from the other Praxis II titles compared in this report (100-200), 
which makes comparative interpretations difficult. 

8 This was calculated by comparing the means within race/ethnicity groups rather than between 

race/ethnicity groups. 

9 See http://www.ets.org/praxis/prepare/materials/test_prep to download ETS TAAGs. 

10 The names of interested faculty members were quickly conveyed to the Praxis test 
development leaders for possible recruitment as item writers and reviewers, and several have 
already been engaged to review items. 

1 1 See Appendix G for an overview by Lloyd Bond, Senior Scholar at the Carnegie Foundation 
for the Advancement of Teaching, on how students process the questions on a Praxis exam. 
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Appendix A 

Primer on Standard Setting 

A standard-setting study produces a passing score recommendation. A passing score is 
the minimum test score that a test-taker needs to achieve to pass the particular licensure test and 
be awarded a license to teach. Each state sets its own passing score. ETS does not set passing 
scores; that is, the responsibility of each state’s licensing agency. 

Standard-setting studies serve two purposes. First, they are designed to identify the level 
of knowledge for a teacher candidate to be considered minimally qualified for independent, 
beginning practice. The level of knowledge is represented by a minimum test score candidates 
need to achieve. Second, the studies are designed to reconfirm the relevance (validity) of the test 
content for teachers in the adopting state. 

Different standard-setting approaches are used for different test structures. That is, there 
is a preferred standard-setting method for multiple choice (MC) test items and another for 
constructed response (CR) test items. ETS recommends and implements a modified Angoff 
method for MC items and a Benchmark method for CR items, both of which are defined later in 
this appendix. One or more ETS standard- setting specialists conduct and facilitate each standard- 
setting study. For each study, a technical report is produced that describes the selection and 
representativeness (i.e., balance of backgrounds by gender, race/ethnicity, and other 
characteristics) of the participants involved and summarizes the standard- setting methods and 
results. 



Panel Formation 

For each method, the state (licensing agency) selects a panel of teachers and teacher 
educators to serve on the standard- setting panel. ETS works closely with the licensing agency to 
identify the appropriate types and numbers of educators from the state. ETS supplies the 
licensing agency with written descriptions of recommended qualifications and demographic 
characteristics of educators. Panels typically consist of 10 to 15 people, the majority of whom are 
practicing, licensed teachers in the content area covered by the test; teacher educators, who 
prepare teacher candidates, often are also represented. States are encouraged to select a panel of 
educators that reflect the diversity in the state (e.g., racial/ethnic, gender, geographic, and 
setting). 
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ETS reviews the nominations and identifies those panelists who meet the criteria. The 
state licensing agency is then asked to confirm and approve the panel composition. ETS 
convenes the panel and conducts the study using the method suitable for the type of test being 
reviewed by the panel. 



Angoff Method of Standard Setting 

This method is used for multiple-choice (MC) test items — items for which there is a 
single correct answer. In brief, this method necessitates that each panelist review each test item 
and judge the percentage of a hypothetical group of 100 minimally qualified test-takers who 
would answer the item correctly. For each item, panelists record the percentage (e.g., 10%, 20%, 
30% . . . 90%) of the 100 hypothetical test-takers who would answer the item correctly. The 
judgments for each panelist (across items) are summed, and the average across panelists is 
computed. This average represents the passing score study value. Before rendering their item 
judgments, panelists take the test and self-score it, define the knowledge and skills of minimally 
qualified test-takers, receive appropriate training, and practice making standard- setting 
judgments. Panelists are also asked to verify that the test content is valid for use in that state. 

Benchmark Method of Standard Setting 

ETS uses the Benchmark method for items that require constructed responses. In this 
method, each panelist reviews the item, scoring rubric, and examples of candidates’ responses 
that are clearly illustrative of the scale points on the scoring rubric for an item. Panelists are then 
asked to identify which benchmark response (scale point) is most likely to be earned by a 
minimally qualified test-taker. If a test consists both of MC and constructed-response (CR) 
items, the recommended number of points from the MC section is combined with the points from 
the CR section to arrive at the test-level recommended passing score. For an exclusively CR test, 
the passing score is the average number of points recommended by the panel. 

Before rendering their item judgments, panelists respond to the CR items and self-score 
them, define the knowledge and skills of just-qualified test-takers, and receive appropriate 
training for and practice making standard-setting judgments. Panelists are also asked to verify 
that the test content is valid for use in that state. 
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Standard-Setting Reports 

Approximately 6 weeks after the standard- setting study is completed, the state receives a 
study report documenting who participated, the procedures and methods, and the results. The 
report also includes information about the standard error of the test and passing score 
recommendations within one and two standard errors of the panel’s recommendation. States may 
use this information and other state- specific information to decide on the operational passing 
score. 
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Appendix B 

Description of Selected Background Variables and Classifications 
Used in the Quantitative Analyses of Praxis I and Selected Praxis II Tests 



1. Race/ethnicity. Five general categories of ethnicity were used: White, African 
American, Hispanic, Asian American, and Native American. White and African 
American candidates were used in the predictive analyses. 

2. Undergraduate grade point average (UGPA) was reported in six categories, five of 
which were used in this report: 1.5-1.99, 2.0-2.49, 2.5-2.99, 3.0-3.49, and 3. 5-4.0. 
The two condensed categories were 3.0 and greater, and less than 3.0. 

3. Teacher education program enrollment status consisted of three types: current, 
formerly, and never. In predictive modeling, the comparison is made between current 
and not current (i.e., formerly and never). 

4. Candidate educational attainment level was reported in nine categories, eight of 
which were initially used in the analyses: freshman, sophomore, junior, senior, earned 
bachelor’s degree, bachelor’s + credits, earned master’s degree, and master’s + 
credits. The reduced categories were less than a college senior, and college senior and 
higher. 

5. Parent educational attainment (proxy for SES) was the higher educational attainment 
level between both parents based on the following categories: some high school or 
less, high school diploma, some post- secondary education, associate degree, 
bachelor’s degree, some graduate or professional school, and graduate or professional 
degree. The two condensed categories were less than a bachelor’s degree, and 
bachelor’s degree and above. 

6. Undergraduate major. Close to 100 categories exist, so these were consolidated to 
represent domains similar to those found in the career interest research literature 
(Emmerich, Rock, & Trapani, 2004). These groupings were science, technology, 
business, social science, education, and humanities. For predictive modeling, 
noneducation majors were one group and education majors were the other group. 
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7. Institutional selectivity status. Based on the 2009 Barron ’s Profile of American 
Colleges ratings of schools, six different classifications were used: noncompetitive, 
less competitive, competitive, competitive +, very competitive, and very competitive 
+ (included most competitive and most competitive + as well). In the predictive 
models, competitive and lower selective schools were one group and those at 
competitive + and higher were the other group. 
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Appendix C 
Statistical Formulas 

The statistics displayed in the data tables throughout the report included traditional 
descriptive statistics such as category frequencies and percentages, test score means, and 
standard deviations. Effect sizes (Cohen, 1988) were computed to reflect differences between 
groups on a particular variable in standardized rather than absolute terms. The formula for effect 
size is: 



d = Mi - M 2 I Spooled (Cl) 

where Mi and M 2 represent the mean scores for the two groups being compared and c poo i e d 
represents the pooled standard deviation between the two groups, which is represented as the 
root mean square of the two standard deviations. The magnitude of the effect size was grouped 
into the following categories: small (0.20-0.49), medium (0.50-0.79), large (0.80 and above). 

Linear regression analyses were performed using the Praxis I or selected Praxis II scale 
score as the dependent variable and the re-coded background variables as independent variables. 
The analyses were conducted in a stepwise fashion, so that the unique contribution of each 
additional variable could be assessed in order to determine whether its effect in predicting Praxis 
I or selected Praxis II scale scores was significant. First, a test for the overall significance of the 
regression model was needed to assess the adequacy of the model. 

The relative significance of each predictor variable was determined by looking at a ratio 
showing the relation between the regression weight (beta) and its standard error, called a t- 
statistic: 



t = p/ SE($) (C2) 

This value was then compared to a value of approximately + 1.96, where larger values indicated 
a nonzero or significant effect in predicting the Praxis I or selected Praxis II scale score. Given 
differing scale score distributions across the Praxis I and selected Praxis II tests, standardized 
regression coefficients helped to further explain the effects of the predictor variables. The 
formula to obtain such values was 



P ' = r X x X ryx 



(C3) 
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where r xx 1 represents the inverse of the inter-correlation matrix of the predictor variables and ryx 
represents the inter-correlation matrix of each predictor variable with the dependent variable, 
Praxis I or selected Praxis II scale score. These two matrices were multiplied together to get the 
standardized regression coefficients. 
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Appendix D 

Summary of Regression Analyses Using Praxis I and Selected Praxis II Tests 
and Background Information Questionnaire (BIQ) Variables 



Table D1 



Summary of Full Regression Results for Praxis I Reading 







Stepwise selection 




Parameter estimates 




Step 


Predictor 


2 

r 


Total r 2 


P 


SE 


Sig. 


Std. p 


1 


Race/ethnicity 


0.10 


0.10 


-2.84 


0.31 


-9.20 


< .01 


2 


Major 


0.05 


0.15 


-2.57 


0.08 


-33.21 


< .01 


3 


UGPA 


0.04 


0.19 


2.73 


0.07 


37.32 


< .01 


4 


Selectivity 


0.02 


0.21 


1.64 


0.06 


25.23 


< .01 


5 


Parent highest ed. 


<.01 


0.22 


0.71 


0.06 


11.05 


< .01 


6 


Race/ethnicityx candidate 
highest ed. 


<.01 


0.22 


-2.59 


0.22 


-11.69 


< .01 


7 


Teacher ed. program 
enrollment 


<.01 


0.22 


0.48 


0.06 


7.47 


< .01 


8 


Candidate ed. level 


<.01 


0.22 


0.53 


0.08 


7.02 


< .01 


9 


Race/ethnicityx major 


<.01 


0.22 


-1.36 


0.24 


-5.56 


< .01 


10 


Race/ethnicityx parent 
highest ed. 


<.01 


0.22 


0.65 


0.21 


3.08 


< .01 


11 


Race/ethnicityx UGPA 


<.01 


0.22 


-0.59 


0.21 


-2.85 


< .01 


12 a 


Race/ethnicityx teacher ed. 
program enrollment 


<.01 


0.22 


-0.45 


0.21 


-2.12 


0.03 



Note. Ed. = education; Sig. = significance of regression coefficient; Std. P = standardized 



regression coefficient; UGPA = undergraduate grade point average. 

a The interaction between race/ethnicity and selectivity was not significant and was dropped from 
the model at Step 13. 
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Table D2 



Summary of Full Regression Results for Praxis I Writing 







Stepwise selection 




Parameter estimates 


Step 


Predictor 


2 

r 


Total r 2 


P 


SE 


Sig. 


Std. p 


1 


UGPA 


0.07 


0.07 


2.24 


0.06 


< .01 


0.23 


2 


Race/ethnicity 


0.05 


0.12 


-2.18 


0.14 


< .01 


-0.14 


3 


Selectivity 


0.04 


0.16 


1.58 


0.05 


< .01 


0.17 


4 


Major 


0.02 


0.18 


-1.48 


0.05 


< .01 


-0.15 


5 


Parent highest ed. 


0.01 


0.19 


0.82 


0.05 


< .01 


0.09 


6 


Race/ethnicity x candidate 
ed. level 


<.01 


0.19 


-0.90 


0.15 


< .01 


-0.04 


7 


Race/ethnicityx UGPA 


<.01 


0.19 


-0.85 


0.16 


< .01 


-0.04 


8 a 


Teacher ed. program 
enrollment 


<.01 


0.19 


0.23 


0.05 


< .01 


0.03 



Note. When interactions of race/ethnicity with another BIQ variable were significant and the 
main effect of the BIQ variable was not significant, the main effect was not re-inserted into the 
final model as typically done in regression analysis. This was true in the case of candidate 
education level. Ed. = education, Sig. = significance of regression coefficient, Std. P = 
standardized regression coefficient, UGPA = undergraduate grade point average. 
a Candidate education level and the interactions of ethnicity with enrollment in a teacher 
education program, parent highest education, major, and selectivity respectively were not 
significant and were dropped from the model at Step 9. 
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Table D3 



Summary of Full Regression Results for Praxis I Mathematics 







Stepwise selection 




Parameter estimates 




Step 


Predictor 


2 

r 


Total r 2 


P 


SE 


Sig. 


Std. p 


1 


Race/ethnicity 


0.11 


0.11 


-3.55 


0.38 


< .01 


-0.14 


2 


Selectivity 


0.03 


0.14 


2.36 


0.09 


< .01 


0.15 


3 


UGPA 


0.03 


0.16 


2.83 


0.09 


< .01 


0.17 


4 


Major 


0.01 


0.17 


-1.86 


0.10 


< .01 


-0.11 


5 


Candidate ed. level 


0.01 


0.18 


-1.10 


0.10 


< .01 


-0.07 


6 


Parent highest ed. 


< .01 


0.18 


0.92 


0.08 


< .01 


0.06 


7 


Race/ethnicity x UGPA 


< .01 


0.18 


-1.89 


0.27 


<.01 


-0.05 


8 


Race/ethnicity x 
candidate ed. level 


<.01 


0.18 


-1.84 


0.29 


< .01 


-0.06 


9 


Teacher ed. program 
enrollment 


<.01 


0.18 


0.48 


0.08 


< .01 


0.03 


10 


Race/ethnicityx teacher 
ed. program enrollment 


<.01 


0.18 


-0.86 


0.27 


< .01 


-0.02 


11 


Race/ethnicityx major 


< .01 


0.18 


-0.92 


0.31 


< .01 


-0.03 


12 a 


Race/ethnicityx 

selectivity 


<.01 


0.18 


-0.91 


0.32 


< .01 


-0.02 



Note. Ed. = education, Sig. = significance of regression coefficient, Std. P = standardized 



regression coefficient, UGPA = undergraduate grade point average. 

a The interaction between race/ethnicity and parent highest education was not significant and was 
dropped from the model at Step 13. 
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Table D4 



Summary of Regression Results for Praxis II Elementary Education Tests 





Elementary education: Curriculum, 
instruction, & assessment 




Elementary education: 
Content knowledge 




Predictor 


3 


SE 


Sig. 


Std. p 


2 

r 


P 


SE 


Sig. 


Std. p 


2 

r 


Race/ 

Ethnicity 


-14.20 


.28 


< .01 


-.21 


.04 


-17.44 


.21 


< .01 


-.30 


.09 


UGPA 


7.17 


.19 


< .01 


.15 


.02 


7.01 


.16 


< .01 


.16 


.03 


Major 


-3.27 


.16 


< .01 


-.09 


.01 


-4.67 


.13 


< .01 


-.13 


.02 


Selectivity 


3.30 


.12 


< .01 


.11 


.01 


5.96 


.12 


< .01 


.17 


.03 



Note. Please refer to Appendix C for more information on the formulas used for this table. 



Sig. = significance of regression coefficient, Std. P = standardized regression coefficient, 
UGPA = undergraduate grade point average. 



Table D5 



Summary of Regression Results for Praxis II Mathematics Tests 





Middle school mathematics: 
Content knowledge 






Mathematics: 
Content knowledge 




Predictor 


P 


SE 


Sig. 


Std. p 


2 

r 


P 


SE 


Sig. 


Std. p 


2 

r 


Race/ 

Ethnicity 


-15.35 


.48 


< .01 


-.23 


.05 


-23.63 


.61 


<.01 


-.29 


.08 


UGPA 


3.93 


.36 


<.01 


.08 


.01 


8.12 


.45 


<.01 


.13 


.02 


Major 


-2.70 


.27 


<.01 


-.07 


.01 


-2.73 


.34 


< .01 


-.06 


.00 


Selectivity 


5.47 


.26 


< .01 


.15 


.02 


9.92 


.34 


<.01 


.22 


.05 



Note. Please refer to Appendix C for more information on the formulas used for this table. 



Sig. = significance of regression coefficient, Std. P = standardized regression coefficient; 
UGPA = undergraduate grade point average. 
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Table D6 

Summary of Regression Results for Praxis II Social Studies and English Tests 



English language, literature, and 
Social studies: Content knowledge composition: Content knowledge 



Predictor 


3 


SE 


Sig. 


Std. p 


2 

r 


3 


SE 


Sig. 


Std. p 


IT 

r 


Race/ 

Ethnicity 


-13.38 


.41 


< .01 


-.21 


.04 


-18.04 


.34 


< .01 


-.32 


.10 


UGPA 


6.30 


.24 


< .01 


.17 


.03 


7.06 


.26 


< .01 


.16 


.03 


Major 


-3.81 


.20 


< .01 


-.12 


.02 


-4.09 


.18 


< .01 


-.13 


.02 


Selectivity 


4.32 


.19 


< .01 


.14 


.02 


5.06 


.18 


< .01 


.17 


.03 



Note. Please refer to Appendix C for more information on the formulas used for this table. 



Sig. = significance of regression coefficient, Std. P = standardized regression coefficient, 
UGPA = undergraduate grade point average. 



68 




Appendix E 

Script Inviting Campuses to Participate in the Joint Research Project 
of the National Education Association (NEA) and ETS to Understand 
Achievement Gaps in Teacher Licensure and Certification Tests 

Good morning or afternoon. My name is . I am calling on behalf of the 

National Education Association and Educational Testing Service. The National Education 
Association (largest teacher union in the United States) and the Educational Testing Service 
(world’s largest testing and measurement company) are involved in a joint research project 
focused on achievement gaps among teachers seeking licensure. To that end, we would like to 
invite your campus to participate in this research by helping us to understand achievement gaps 
in teacher licensure tests and to identify ways to narrow these gaps. Your campus has been 
selected because our information suggests that 

• you graduate significant numbers of minority teacher candidates, 

• your institution is NCATE-accredited, and 

• the state where you are located requires your students to take the Praxis series of tests 
for licensure. 

Here’s what we plan to do as part of the project we are asking your campus to participate in. 

Using data from ETS’s Praxis series of teacher licensure tests, NEA and ETS have agreed 
to pursue research jointly to answer the following questions: 

• Where do the achievement gaps lie on Praxis? 

• Is the gap the same on all of the highest- volume tests? 

• What are the characteristics of people who do well or poorly on these tests? 

• How can ETS and NEA intervene to narrow the gaps? 

The project is divided into four parts: 

1. Preliminary research/analysis of the Praxis I and Praxis II tests 

2. Site research at higher education institutions using Praxis I for entrance into teacher 
preparation programs and Praxis II for teacher licensure 

3. Analysis of results and recommendations for interventions 
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4. Publicizing results of the study and recommended interventions 

Preliminary Research and Analysis 

• Identify highest volume Praxis II tests (paper based and computer based) with the 
most significant performance gaps. 

• Analyze paper-based and computer-based data from Praxis I tests in mathematics, 
reading, and writing to yield most significant performance gaps. 

• Characterize testing candidates with low performance on both sets of tests (from 
performance data and biographical data collected at time of registration) 

Provided you agree to participate, the researchers and educators involved in this project will visit 
your campus for the following purposes: 

• To work with teacher education faculty to examine local indicators of student 
performance 

• To identify test areas where minorities have the greatest problems 

• To identify interventions that have been used successfully to address the problem 
areas 

• To examine linkages between schools of education and schools of arts and sciences 
designed to supplement content provided by school of education 

• To investigate campus practices that have been known to improve performance 

• To investigate how the curriculum is designed and revised to address identified 
problems 

No confidential information will be released. 

The information we obtain from you and your colleagues will be analyzed and included in future 
plans for interventions and in reports to be disseminated nationally. 

Timeframe to Visit Campuses 

Preliminary research/analysis: 3 months (August, September, October) 

Site visits: spread over a 6-month period (March, April, May, August, September and 
October 2006) 
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We would like to speak with you and members of your teaching staff who are involved in 
preparing teachers. Ideally, we would spend an afternoon and a morning on your campus, 
speaking with teacher education faculty as a group and with some particularly key people 
individually. We would ask you to identify who those key faculty members would be. 

In the hope that you are willing to have your campus participate, I’d like to schedule a 
time for our visit. Our group of visitors will be three or four people. Can you give me a few dates 
when our visit would be convenient for you? 
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Appendix F 

National Education Association (NEA)/Educational Testing Service (ETS) 

Interview Guides 

The following provides a summary of the interviews conducted between NEA, ETS, and 
four different audiences within each of seven schools selected for qualitative analysis. These 
audiences included deans, department chairs, university faculty, and students. The interview 
guides were divided into four main sections: an introduction, an overview of the school or 
department, questions about achievement gaps, and general questions about Praxis tests. All 
interviews concluded with final thoughts and a closing. 

Introduction (Common to All Audiences) 

Representatives from the National Education Association (NEA) and Educational Testing 
Service (ETS) are talking with administrators and faculty in colleges of education to discuss 
achievement gaps between the majority Praxis test-takers and those from African American, 
Hispanic, and Native American populations. The objective of the research is to understand key 
issues related to this gap and, in an effort to develop solutions, identify and understand efforts 
that have been effective in closing the gaps. 

Thank you for taking the time to talk with us. 

Overview: Part 1 (Deans Only) 

• What is the basic structure of your education program? 

• What academic programs are included? 

• What are the requirements for admission? 

• What core courses must all students take? 

• In what areas do you graduate most students? 

• Are alternate route programs available? 

• What is the current enrollment in the school of education? 

• What percentage of current enrollees are Caucasian . ., Asian 

American,. ., African American ., Hispanic _, and 

Native American ? 
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• Approximately, how many students from each group graduate from your program 

each year? Caucasian Asian American , African American 

Hispanic and Native American. ? 

• Our records indicate that your college/school has been very successful in graduating 
students from African American, Hispanic, and/or Native American populations. 
What would you say have been your “keys to success”? 

Overview: Part 2 (To All Audiences but Students) 

Deans 

How do you measure success of students for your college/school? 

• How often are students assessed? 

• What kinds of assessments are given? Who develops them (faculty, administration, 
outside vendor)? 

• How is the assessment information used? As you are using the assessment 
information, do you segment the data by ethnic group and examine results separately? 

Department Chairs/Center Directors 

Describe the offerings of your department/center, including information about core courses 

required for majors, etc. 

How do you measure success of students for your center/department? 

• How often are students assessed? 



• What kinds of assessments are given? Who develops them (faculty, administration, 
outside vendor)? 

• How is the assessment information used? As you are using the assessment 
information, do you segment the data by ethnic group and examine results separately? 



Faculty 

• How do you use available institutional assessment data in your classes? 
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Deans 



Achievement Gaps (All Audiences but Students) 



• Have you identified any subjects and/or skill areas required for success in teaching 
that are more problematic for students from African American, Hispanic, and Native 
American populations than for those from the majority group? Please describe what 
aspects of these areas tend to show the largest gaps and explain why you think the 
differences exist. 

• Have you identified any subjects and/or skill areas required for success in teaching 
that seem to be problematic for all students, with few observed differences between 
ethnic groups? Please describe what aspects of these areas tend to present more 
problems and explain why this area presents difficulties for students in general. 

• Please describe any programs or interventions that are specifically designed to 
improve the performance of students who are having trouble mastering the required 
materials. Describe the successes that these programs/interventions have had. (If not 
addressed, PROBE — how have you assessed this program/intervention?) 

• Have any of these programs or interventions been particularly successful in boosting 
performance of students from African American, Hispanic, and Native American 
populations and thereby reducing the gap? Please describe how they have achieved 
this success. 

• Have changes been made to the curriculum with the aim of improving student 
performance, especially the performance of African American, Hispanic, and Native 
American students? 

• If yes, what changes have been made and how successful have they been? 

• Does the college/school of education work with other academic programs within your 
university to help improve minority student performance? 

• If yes, what specifically is being done? 

• Can you identify education policies within the last 5 years that have affected your 
teacher education program? If so, explain how. 
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• How have state policies about teacher testing affected your institution’s success in the 
preparation of new teachers, especially those who will work in low performing 
schools? Please provide an example. 

• Does your institution participate in item-development and item-review processes 
related to the Praxis tests? If so, who from your institution serves in this capacity? 

Has it been helpful in guiding your teacher preparation program? 

• What suggestions would you have for our organizations as we attempt to develop 
programs designed to close the achievement gaps between the majority populations 
and the three ethnic groups? 

Department Chairs and Center Directors 

• Have you identified any subjects and/or skill areas required for success in teaching 
that are more problematic for students from African American, Hispanic, and Native 
American populations than for those from the majority group? Please describe what 
aspects of these areas tend to show the largest gaps and explain why you think the 
differences exist. 

• Have you identified any subjects and/or skill areas required for success in teaching 
that seem to be problematic for all students, with few observed differences between 
ethnic groups? Please describe what aspects of these areas tend to present more 
problems and explain why this area presents difficulties for students in general. 

• Please describe any programs or interventions that are specifically designed to 
improve the performance of students who are having trouble mastering the required 
materials. Describe the successes that these programs/interventions have had. (If not 
addressed, PROBE — how have you assessed this program/intervention?) 

• Have any of these programs or interventions been particularly successful in boosting 
performance of students from either African American, Hispanic, and Native 
American populations and thereby reducing the gap? Please describe how they have 
achieved this success. 
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• Have changes been made to the curriculum with the aim of improving student 
performance, especially the performance of African American, Hispanic, and Native 
American students? 

• If yes, what changes have been made and how successful have they been? 

• Does the center/department work with other academic programs within your 
university to help improve minority student performance? 

• If yes, what specifically is being done? 

• What suggestions would you have for our organizations as we attempt to develop 
programs designed to close the achievement gaps between the majority populations 
and the three ethnic groups? 

Faculty 

• Have you identified any subjects and/or skill areas required for success in teaching 
that are more problematic for students from African American, Hispanic, and Native 
American populations than for those from the majority group? Please describe what 
aspects of these areas tend to show the largest gaps and explain why you think the 
differences exist. 

• Have you identified any subjects and/or skill areas required for success in teaching 
that seem to be problematic for all students, with few observed differences between 
ethnic groups? Please describe what aspects of these areas tend to present more 
problems and explain why this area presents difficulties for students in general. 

• Please describe any programs or interventions that are specifically designed to 
improve the performance of students who are having trouble mastering the required 
materials. Describe the successes that these programs/interventions have had. (If not 
addressed, PROBE — how have you assessed this program/intervention?) 

• Have any of these programs or interventions been particularly successful in boosting 
performance of students from African American, Hispanic, and Native American 
populations and thereby reducing the gap? Please describe how they have achieved 
this success. 
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Have changes been made to the curriculum with the aim of improving student 
performance, especially the performance of African American, Hispanic, and Native 
American students? 

• If yes, what changes have been made and how successful have they been? 

What suggestions would you have for our organizations as we attempt to develop 
programs designed to close the achievement gaps between the majority populations 
and the three ethnic groups? 

Praxis Tests (All Audiences) 

Does the college/school of education currently have any programs that are 
specifically designed to help students prepare for Praxis tests? 

• If yes, please describe what is being done. 

What is the college/school of education faculty’s role in preparing students for the 
Praxis tests? 

To what extent have students and faculty from your college/school of education 
participated in the ETS Praxis Workshop Program? 

To your knowledge, what percentage of the faculty in your college/school of 
education has taken the Praxis test themselves? 

How are you using the Institutional Score Report that ETS provides to inform your 
Praxis preparation and intervention strategies? PROBE if not mentioned: We are 
particularly interested in how you use the detailed category score information and the 
detailed score information by quartile. Can you describe how you use these sections 
of the report? 

To what extent do faculty in your college/school of education make use of the 
resources on the ETS website, specifically Tests at a Glance (TAAGs)? Are they used 
elsewhere in your college/school of education? If yes, please describe how. 




• To what extent do faculty in your college/school of education make use of the Praxis 
Study Guides? Are they used elsewhere in your college/school of education? If yes, 
please describe how. 

• To what extent do faculty in your college/school of education make use of the Praxis 
Diagnostic Preparation Program? Are they used elsewhere in your college/school of 
education? If yes, please describe how. 

Department Chairs and Center Directors 

• Does the center/department currently have any programs that are specifically 
designed to help students prepare for Praxis tests? 

• If yes, please describe what is being done. 

• What is the center/department faculty’s role in preparing students for the Praxis tests? 

• To what extent have students and faculty from your center/department participated in 
the ETS Praxis Workshop Program? 

• To your knowledge, what percentage of the faculty in your center/department has 
taken the Praxis test themselves? 

• To what extent do faculty in your center/department make use of the resources on the 
ETS website, specifically Tests at a Glance (TAAGs)? Are they used elsewhere in 
your center/department? If yes, please describe how. 

• To what extent do faculty in your center/department make use of the Praxis Study 
Guides? Are they used elsewhere in your center/department? If yes, please describe 
how. 

• To what extent do faculty in your center/department make use of the Praxis 
Diagnostic Preparation Program? Are they used elsewhere in your center/department? 
If yes, please describe how. 

• How are you using the Institutional Score Report that ETS provides to inform your 
Praxis preparation and intervention strategies? PROBE if not mentioned: We are 
particularly interested in how you use the detailed category score information and 
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detailed score information by quartile. Can you describe how you use these sections 
of the report? 



Faculty 

• Do you do anything in your classes that is specifically designed to help students 
prepare for Praxis tests? 

• If yes, please describe. 

• If yes, do you prepare and present the materials? If not, where do you find the 
content? 

• Have you participated in the ETS Praxis Workshop Program? If yes, how useful was 
it to you and how can it be improved? 

• Have you taken the Praxis exam? Has exposure to the test assisted you in preparing 
your students for it? Please explain. 

• Do you make use of the resources on the ETS website, specifically Tests at a Glance 
(TAAGs)? If yes, please describe how you use these resources. How useful have you 
found them to be? How can they be improved? 

• Do you make use of the Praxis Study Guides? If yes, please describe how you use 
these resources. How useful have you found them to be? How can they be improved? 

• Do you make use of the Praxis Diagnostic Preparation Program? If yes, please 
describe how you use these resources. How useful have you found them to be? How 
can they be improved? 

• How are you using the Institutional Score Report that ETS provides to inform your 
Praxis preparation and intervention strategies? PROBE if not mentioned: We are 
particularly interested in how you use the detailed category score information and 
detailed score information by quartile. Can you describe how you use these sections 
of the report? 



79 




Students 

• How many of you have had both Praxis I & II? All the rest of you have taken Praxis I, 
right? 

• What steps did you take to prepare yourself for the Praxis test(s)? 

• Do you make use of the resources on the ETS website, specifically Tests at a Glance 
(TAAGs)? If yes, please describe how you use these resources. How useful have you 
found them to be? How can they be improved? 

• What support did you receive in preparing to take the test? 

• What additional support would you like to have had? 

• How comfortable were you when you took the Praxis test? Did you feel confident that 
you could handle the material on the test? Or did you feel anxious about it? 

• For those you who felt anxious about the test, what do you think could have helped to 
reduce your anxiety? 

• What areas of the test(s) did you find were most difficult? 

• Why do you think there is an achievement gap between minority and White students? 

• What suggestions would you have for our organizations as we attempt to develop 
programs designed to close the achievement gaps between the majority populations 
and the three ethnic groups? 

Final Thoughts and Other Issues (All Audiences) 

Before we conclude our discussion, are there any other issues or challenges that the 
college/school of education is facing, especially in relation to the performance of African- 
American, Hispanic, and Native American students that we haven’t discussed? 

• If yes, what are they and how are you meeting those challenges? 

Closing (All Audiences) 

Thank you very much for your time. Please feel free to contact us if you have any questions or 
would like to continue our discussion. 
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Appendix G 

Lloyd Bond Findings and Recommendations 

An HBCU steering committee, made up of 10 HBCU presidents, commissioned Lloyd 
Bond, Senior Scholar at the Carnegie Foundation for the Advancement of Teaching, to visit 
several HBCU campuses in 2003-2004 in order to investigate student performance on Praxis 
tests. Bond and colleagues administered Praxis items informally to a group of students on each 
campus. Following the administration, he asked the students to describe their solution strategies, 
in order to better understand the ways that African American students process test content. 
Specifically, the intent was to examine the extent of students’ declarative (recall of facts) and 
procedural (application) knowledge and skills in key areas measured by Praxis I. 

Bond’s summary findings and recommendations from his previously unpublished 
research appear below. 



Findings 

• Students’ responses to sample items indicated the typical problems associated with 
taking standardized tests: test anxiety; lack of time-management skills; lack of 
familiarity with the content, format, and purpose of standardized tests; a tendency to 
either misread items, read too much into items, or failure to read items fully and 
carefully; and a tendency to miss key words and phrases. 

• For reasons not yet completely understood, the knowledge that many students 
possessed about arithmetic, algebra, and basic geometry was largely inert knowledge 
that was unusable in problem solving. While several students could recall many of the 
basic concepts associated with the series of problems used in the exercise, there 
appeared to be a disconnect between their ability to recall facts and their ability to 
bring the concepts forward in an integrated way to perform basic math operations. 

• Many students who were unsure of their own knowledge and skill, substituted “close 
enough” for the “correct solution.” In other words, many confused estimation 
problems with problems that required a unique, correct answer. 

• Many students were seriously inefficient in their test-taking strategies. They tended to 
spend inordinate amounts time on subproblems of the main problem that should have 
required seconds of mental arithmetic instead of becoming major digressions. 
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Students were not comfortable with mental arithmetic. Most students relied on a step- 
by-step approach to solving math problems. This strategy is probably a direct result of 
the culture of teaching that has traditionally valued a student’s ability to demonstrate 
an understanding of all steps required to solve a math problem. This approach, 
however, runs counter to the testing culture, which requires efficiency in responding 
to math items. For example, students struggled with items requiring no more than 
estimation or rounding and they tended to know only one way to solve a math 
problem. Many of the students were so far removed from their high school math 
courses (and so little practiced in routinely doing mental calculations) that they had 
not “automated” simple procedures such as the times tables. 

• Frequently, we encountered considerable anger at the entire process — some of it 
directed at the test, some directed at the faculty and institution, and some simply 
diffused. There may be no easy solution to this problem, but students must be made to 
realize that the tests are a hurdle that they themselves must jump. Blaming others will 
get them nowhere. 



Recommendations 

Bond urged institutions to find ways to strengthen their students’ basic skills in math, 
writing, grammar, and reading comprehension that are needed for Praxis I. Additionally, he felt 
that students must be made to understand that facility with these skills is not optional if they wish 
to teach. In relation specifically to testing, he found that students needed more practice with the 
kinds of items included in the tests they will take and that they needed to develop realistic views 
about the purposes, content, format, and demands of these standardized tests. It is essential, he 
noted, that educators find ways to help students internalize commonly known test-taking skills — 
legitimate cognitive abilities, not tricks — that have grown out of the paper- and-pencil testing 
culture. The students’ reliance on a step-by-step approach to math problems, failure to mark key 
words and phrases in reading passages, lack of guessing strategies, etc., are indicative of the need 
for such internalization. Finally, Bond called for a deepening of faculty understanding of the 
tests, resulting ultimately in solutions at the curriculum and faculty level. 
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